Downloading multiple files with requests in Python

心不动则不痛 提交于 2021-02-10 23:42:19

问题


Currently im facing following problem:

I have 3 download links in a list. Only the last file in the list is downloaded completely. The others have a file size of one kilobyte.

Code:

from requests import get

def download(url, filename):
    with open(filename, "wb") as file:
        response = get(url, stream=True)
        file.write(response.content)

for link in f:
    url = link
    split_url = url.split("/")
    filename = split_url[-1]
    filename = filename.replace("\n", "")
    download(url,filename)

The result looks like this:

Result

How do I make sure that all files are downloaded correctly? All links are direct download links.

Thanks in advance!

EDIT:

I discovered it only happens when I read the links from the .txt

If I create the list in python like this:

links = ["http://ipv4.download.thinkbroadband.com/20MB.zip",
            "http://ipv4.download.thinkbroadband.com/10MB.zip",
            "http://ipv4.download.thinkbroadband.com/5MB.zip"]

... the problem doesnt appear.

reproduceable example:

from requests import get

def download(url, filename):
    with open(filename, "wb") as file:
        response = get(url, stream = True)
        file.write(response.content)

f = open('links.txt','r')
for link in f:
    url = link
    split_url = url.split("/")
    filename = split_url[-1]
    filename = filename.replace("\n", "")
    download(url,filename)

content of links.txt:

http://ipv4.download.thinkbroadband.com/20MB.zip
http://ipv4.download.thinkbroadband.com/10MB.zip
http://ipv4.download.thinkbroadband.com/5MB.zip

回答1:


url = url.replace("\n", "")

solved it!



来源:https://stackoverflow.com/questions/60364827/downloading-multiple-files-with-requests-in-python

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!