urllib2 urlopen works very randomly

Question

For some reasons this part where I fetch JSON data from following url will only works sometimes. And sometimes it will return 404 error, and complain about missing header attribute. It will work 100% of the time if I paste it onto a web browser. So I'm sure the link is not broken or something.

I get the following error in Python:

AttributeError: 'HTTPError' object has no attribute 'header'

What's the reason for this and can it be fixed? Btw I removed API key since it is private.

try:
    url = "http://api.themoviedb.org/3/search/person?api_key=API-KEY&query=natalie+portman"
    header = { 'User-Agent': 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.16) Gecko/20110319 Firefox/3.6.16' }
    req = urllib2.Request(url, None, header)
    f = urllib2.urlopen(req)
except urllib2.HTTPError, e:
    print e.code
    print e.msg
    print e.header
    print e.fp.read()

Answer 1

As is documented here, you need to explicitly accept JSON. Just add the second line after the first.

req = urllib2.Request(url, None, header)
req.add_header('Accept', 'application/json')