How do I send a custom header with urllib2 in a HTTP Request?

Question

I want to send a custom "Accept" header in my request when using urllib2.urlopen(..). How do I do that?

Answer 1

Not quite. Creating a Request object does not actually send the request, and Request objects have no Read() method. (Also: read() is lowercase.) All you need to do is pass the Request as the first argument to urlopen() and that will give you your response.

import urllib2
request = urllib2.Request("http://www.google.com", headers={"Accept" : "text/html"})
contents = urllib2.urlopen(request).read()

Answer 2

I normally use:

import urllib2

request_headers = {
"Accept-Language": "en-US,en;q=0.5",
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:40.0) Gecko/20100101 Firefox/40.0",
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
"Referer": "http://thewebsite.com",
"Connection": "keep-alive" 
}

request = urllib2.Request("https://thewebsite.com", headers=request_headers)
response = urllib2.urlopen(request).read()
print(response)

Answer 3

Beside the other solutions mentioned already, you could use add_header method.

So the example provided py @pantsgolem will be:

import urllib2
request = urllib2.Request("http://www.google.com")

request.add_header('Accept','text/html')

##Show the header having the key 'Accept'
request.get_header('Accept')

response = urllib2.urlopen(request)
response.read()