urllib2 returns 404 for a website which displays fine in browsers

Question

I am not able to open one particular url using urllib2. Same approach works well with other websites such as \"http://www.google.com\" but not this site (which also displays

Answer 1

I just tried this and received 404 code and page back.

At a guess it's doing User-Agent detection which either by accident or on purpose doesn't serve content to python urllib.

Clarification, with urllib, I received the urlopen returned a response object with a 404 code and HTML content. With urllib2.urlopen an urllib2.HTTPError exception was raised.

I'd suggest you try setting your User Agent to something that looks like a browser. There's a question about this here: Changing user agent on urllib2.urlopen