How to limit download rate of HTTP requests in requests python library?

你说的曾经没有我的故事 提交于 2019-12-17 18:28:26

问题


Is it possible to limit the download rate of GET requests using the requests Python library? For instance, with a command like this:

r = requests.get('https://stackoverflow.com/')

...is it possible to limit the download rate? I'm hoping for something similar to this wget command:

wget --limit-rate=20k https://stackoverflow.com/

I know it's possible with urllib2. I'm asking specifically about the requests library.


回答1:


There are several approaches to rate limiting; one of them is token bucket, for which you can find a recipe here and another one here.

Usually you would want to do throttling or rate limiting on socket.send() and socket.recv(). You could play with socket-throttle and see if it does what you need.


This is not to be confused with x-ratelimit rate limiting response headers, which are related to a number of requests rather than a download / transfer rate.



来源:https://stackoverflow.com/questions/17691231/how-to-limit-download-rate-of-http-requests-in-requests-python-library

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!