Parallel download using Curl command line utility

后端 未结 8 669
抹茶落季
抹茶落季 2020-12-13 19:08

I want to download some pages from a website and I did it successfully using curl but I was wondering if somehow curl downloads multiple pages at a

8条回答
  •  一向
    一向 (楼主)
    2020-12-13 19:30

    Curl can also accelerate a download of a file by splitting it into parts:

    $ man curl |grep -A2 '\--range'
           -r/--range 
                  (HTTP/FTP/SFTP/FILE)  Retrieve a byte range (i.e a partial docu-
                  ment) from a HTTP/1.1, FTP or  SFTP  server  or  a  local  FILE.
    

    Here is a script that will automatically launch curl with the desired number of concurrent processes: https://github.com/axelabs/splitcurl

提交回复
热议问题