BadStatusLine exception raised when returning reply from server in Python 3

Question

I am trying to port a script to python 3 that submits XML feeds found here:

https://developers.google.com/search-appliance/documentation/files/pushfeed_client.py.txt

Answer 1

After lots of wiresharking, I figured out the cause, and solution, of the problem is the way the content-length header was being set. In my Python 3 port of the script, I copied over the method that set the content-length. Which is this:

headers['Content-length']=str(len(body))

That is incorrect! The correct way would be this:

headers['Content-length']=str(len(bytes(body, 'utf-8')))

Because the payload must be a bytes object. When you bytes encode it, the length is different than the string version.

return urllib.request.Request(theurl, bytes(body, 'utf-8'), headers)

You can safely omit manually setting the content-length header when using anything that derives from http.client.HTTPConnection. It has an internal method that checks for the content-length header, and if it's missing, set it based on the length of the content body, regardless of form.

The issue was a translation but subtle difference between Python 2 and 3 and how it handles strings and encodes them. It must've been some kind of fluke that the regular ASCII version worked when the utf-8 version didn't, oh well.