Changing user agent on urllib2.urlopen

Question

How can I download a webpage with a user agent other than the default one on urllib2.urlopen?

Answer 1

there are two properties of urllib.URLopener() namely:
addheaders = [('User-Agent', 'Python-urllib/1.17'), ('Accept', '*/*')] and
version = 'Python-urllib/1.17'.
To fool the website you need to changes both of these values to an accepted User-Agent. for e.g.
Chrome browser : 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.149 Safari/537.36'
Google bot : 'Googlebot/2.1'
like this

import urllib
page_extractor=urllib.URLopener()  
page_extractor.addheaders = [('User-Agent', 'Googlebot/2.1'), ('Accept', '*/*')]  
page_extractor.version = 'Googlebot/2.1'
page_extractor.retrieve(, )

changing just one property does not work because the website marks it as a suspicious request.