What should I do if socket.setdefaulttimeout() is not working?

Question

I\'m writing a script(multi-threaded) to retrieve contents from a website, and the site\'s not very stable so every now and then there\'s hanging http request which cannot e

Answer 1

From their documentation:

Since Python 2.6, urllib2 uses a .timeout attribute on Request objects internally. However, urllib2.Request has no timeout constructor argument, and urllib2.urlopen() ignores this parameter. mechanize.Request has a timeout constructor argument which is used to set the attribute of the same name, and mechanize.urlopen() does not ignore the timeout attribute.

Perhaps you should try replacing urllib2.Request with mechanize.Request.