Python HTTP Error 505: HTTP Version Not Supported

Question

I have the following code:

import re
from re import sub

import cookielib
from cookielib import CookieJar
import urllib2
from urllib2 import urlopen

cj = CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
opener.addheaders=[('user-agent' , 'Safari/7.0.2')]

def check(word):
    try:
        query = "select * from geo.places where text ='"+word+"'"
        sourceCode=opener.open('http://query.yahooapis.com/v1/public/yql?q='+query+'&diagnostics=true').read()
        print sourceCode
    except Exception, e:
        print str(e)
        print 'ERROR IN MAIN TRY'

myStr = ['I','went','to','Boston']
for item in myStr:
        check(item)

I am trying to query select * from geo.places where text = 'Boston' (for example).

I keep receiving this error:

    HTTP Error 505: HTTP Version Not Supported
    ERROR IN MAIN TRY

What can cause this error and how can I solve it?

Answer 1

Not sure, what is going wrong, but when I try to do the same action using requests library, it works:

>>> import requests
>>> word = "Boston"
>>> query = "select * from geo.places where text ='"+word+"'"
>>> query
"select * from geo.places where text ='Boston'"
>>> baseurl = 'http://query.yahooapis.com/v1/public/yql?q='
>>> url = baseurl + query
>>> url
"http://query.yahooapis.com/v1/public/yql?q=select * from geo.places where text ='Boston'"
>>> req = requests.get(url)
>>> req
<Response [200]>
>>> req.text
u'<?xml version="1.0" encoding="UTF-8"?>\n<query xmlns:yahoo="http://www.yahooapis.com/v1/base.rng" yahoo:count="10" yahoo:created="2014-05-17T21:12:52Z" yahoo:lang="en-US"><results><place xmlns="http://where.yahooapis.com/v1/schema.rng" xml:lang="en-US" yahoo:uri="http://where.yahooapis.com/v1/place/2367105"><woeid>2367105</woeid><placeTypeName code="7">Town</placeTypeName><name>Boston</name><country code="US" type="Country" woeid="23424977">United States</country><admin1 code="US-MA" type="State" woeid="2347580">Massachusetts</admin1><admin2 code="" type="County" woei....

Note, that there are differences, my code is much simpler, it does not work with cookies and it does not try to pretend Safari browser.

If you need to use cookies with requests, you will find very good support for it there.

Answer 2

The URL you construct is not a valid URL. What you send is

GET /v1/public/yql?q=select * from geo.places where text ='I'&diagnostics=true HTTP/1.1
Accept-Encoding: identity
Host: query.yahooapis.com
Connection: close
User-Agent: Safari/7.0.2

There should be no spaces inside the URL, e.g. you have to do proper URL encoding (replace space with '+' etc). I guess requests just fixes the bad URL for you.

Answer 3

Your query might have blank spaces in between. Requests take care of the white spaces in your url and hence you don't have to take care of it. Just replace each " " by "%20" to make the url work.

Answer 4

As stated in other answers, you need to encode your url due to the white space. The call would be urllib.quote if using python2 or urllib.parse.quote for python3. The safe parameter is used to ignore characters when encoding.

from urllib import quote
url = 'http://query.yahooapis.com/v1/public/yql?q=select * from geo.places where text =\'Boston\''
print(quote(url, safe=':/?*=\''))

# outputs "http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20geo.places%20where%20text%20='Boston'"

Answer 5

Use requests is good choose. but we should found out why?

query = "select * from geo.places where text ='"+word+"'" There are some space in your paramter.we should be url encode this space.

you should convert the spaces to '%20', but in python the '%' is special char, you should be escaped use '%%20'