How do i download pdf file over https with python

Question

I am writing a python script, which will save pdf file locally according to the format given in URL. for eg.

https://Hostname/saveReport/file_name.pdf   #saves the content in PDF file.

I am opening this URL through python script :

 import webbrowser
 webbrowser.open("https://Hostname/saveReport/file_name.pdf")  

The url contains lots of images and text. Once this URL is opened i want to save a file in pdf format using python script.

This is what i have done so far.
Code 1:

import requests
url="https://Hostname/saveReport/file_name.pdf"    #Note: It's https
r = requests.get(url, auth=('usrname', 'password'), verify=False)
file = open("file_name.pdf", 'w')
file.write(r.read())
file.close()

Code 2:

 import urllib2
 import ssl
 url="https://Hostname/saveReport/file_name.pdf"
 context = ssl._create_unverified_context()
 response = urllib2.urlopen(url, context=context)  #How should i pass authorization details here?
 html = response.read()

In above code i am getting: urllib2.HTTPError: HTTP Error 401: Unauthorized

If i use Code 2, how can i pass authorization details?

Answer 1

I think this will work

import requests
url="https://Hostname/saveReport/file_name.pdf"    #Note: It's https
r = requests.get(url, auth=('usrname', 'password'), verify=False,stream=True)
r.raw.decode_content = True
with open("file_name.pdf", 'wb') as f:
        shutil.copyfileobj(r.raw, f)      

Answer 2

One way you can do that is:

import urllib3
urllib3.disable_warnings()
url = r"https://websitewithfile.com/file.pdf"
fileName = r"file.pdf"
with urllib3.PoolManager() as http:
    r = http.request('GET', url)
    with open(fileName, 'wb') as fout:
        fout.write(r.data)

Answer 3

For some files - at least tar archives (or even all other files) you can use pip:

import sys
from subprocess import call, run, PIPE
url = "https://blabla.bla/foo.tar.gz"
call([sys.executable, "-m", "pip", "download", url], stdout=PIPE, stderr=PIPE)

But you should confirm that the download was successful some other way as pip would raise error for any files that are not archives containing setup.py, hence stderr=PIPE (Or may be you can determine if the download was successful by parsing subprocess error message).

Answer 4

You can try something like :

import requests
response = requests.get('https://websitewithfile.com/file.pdf',verify=False, auth=('user', 'pass'))
with open('file.pdf','w') as fout:
   fout.write(response.read()):