Multiple simultaneous downloads using Wget?

前端 未结 16 1201
旧时难觅i
旧时难觅i 2020-12-04 04:50

I\'m using wget to download website content, but wget downloads the files one by one.

How can I make wget download using 4 simultaneous connections?

16条回答
  •  無奈伤痛
    2020-12-04 05:14

    use xargs to make wget working in multiple file in parallel

    #!/bin/bash
    
    mywget()
    {
        wget "$1"
    }
    
    export -f mywget
    
    # run wget in parallel using 8 thread/connection
    xargs -P 8 -n 1 -I {} bash -c "mywget '{}'" < list_urls.txt
    

    Aria2 options, The right way working with file smaller than 20mb

    aria2c -k 2M -x 10 -s 10 [url]
    

    -k 2M split file into 2mb chunk

    -k or --min-split-size has default value of 20mb, if you not set this option and file under 20mb it will only run in single connection no matter what value of -x or -s

提交回复
热议问题