Requests with multiple connections

Question

I use the Python Requests library to download a big file, e.g.:

r = requests.get(\"http://bigfile.com/bigfile.bin\")
content = r.content

Th

Answer 1

This solution requires the linux utility named "aria2c", but it has the advantage of easily resuming downloads.

It also assumes that all the files you want to download are listed in the http directory list for location MY_HTTP_LOC. I tested this script on an instance of lighttpd/1.4.26 http server. But, you can easily modify this script so that it works for other setups.

#!/usr/bin/python

import os
import urllib
import re
import subprocess

MY_HTTP_LOC = "http://AAA.BBB.CCC.DDD/"

# retrieve webpage source code
f = urllib.urlopen(MY_HTTP_LOC)
page = f.read()
f.close

# extract relevant URL segments from source code
rgxp = '(\\