Why this request doesn't work?

问题

I want to make a simple stupid twitter app using Twitter API.

If I request this page from my browser it does work:

http://search.twitter.com/search.atom?q=hello&rpp=10&page=1

but if I request this page from python using urllib or urllib2 most of the times it doesn't work:

response = urllib2.urlopen("http://search.twitter.com/search.atom?q=hello&rpp=10&page=1")

and I get this error:

Traceback (most recent call last):
  File "twitter.py", line 24, in <module>
    response = urllib2.urlopen("http://search.twitter.com/search.atom?q=hello&rpp=10&page=1")
  File "/usr/lib/python2.6/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.6/urllib2.py", line 391, in open
    response = self._open(req, data)
  File "/usr/lib/python2.6/urllib2.py", line 409, in _open
    '_open', req)
  File "/usr/lib/python2.6/urllib2.py", line 369, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.6/urllib2.py", line 1161, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/lib/python2.6/urllib2.py", line 1136, in do_open
    raise URLError(err)
urllib2.URLError: <urlopen error [Errno 110] Connection timed out>

Why ??

回答1:

The code seems alright.

The following worked.

>>> import urllib
>>> import urllib2
>>> user_agent = 'curl/7.21.1 (x86_64-apple-darwin10.4.0) libcurl/7.21.1'
>>> url='http://search.twitter.com/search.atom?q=hello&rpp=10&page=1'
>>> headers = { 'User-Agent' : user_agent }
>>> req = urllib2.Request(url, None, headers)
>>> response = urllib2.urlopen(req)
>>> the_page = response.read()
>>> print the_page

The other is twitter actually could not respond. This happens once too often with Twitter.

回答2:

did you change the default socket timeout somewhere in your script? your example code works reliably for me.

it could be your internet connection, or you might try

import socket
socket.setdefaulttimeout(30)

assuming urllib/2 don't override the socket timeout.

来源：https://stackoverflow.com/questions/3964519/why-this-request-doesnt-work

标签

python

urllib2