Bandwidth throttling in Python

时光怂恿深爱的人放手 提交于 2019-12-31 08:39:44

问题


What libraries out there let you control the download speed of network requests (http in particular). I don't see anything built-in in urllib2 (nor in (Py)Qt which I intend on using).

Can Twisted control bandwidth? If not, how can I control the read buffer size of urllib2 or Twisted? sleeping to suspend network operations isn't an option.


回答1:


Of course twisted can. You want twisted.protocols.policies.ThrottlingFactory. Just wrap your existing factory in it before you pass it to whatever wants a factory.




回答2:


urllib2 doesn't offer a way to do this, so you'd have to extend some of the classes it uses and implement rate limiting yourself. You might want to look at this question. If you decide to write a limiter, read up on the token bucket and leaky bucket algorithms.

Alternatively, you could use pycurl along with the CURLOPTMAXRECVSPEEDLARGE option.

EDIT: The urlgrabber package appears to support throttling as well, and is probably easier to understand than pycurl.

If you prefer to program using an event loop model, there's the Twisted approach, which has already been mentioned in another answer.



来源:https://stackoverflow.com/questions/3488616/bandwidth-throttling-in-python

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!