Python Requests, warning: urllib3.connectionpool:Connection pool is full

落爺英雄遲暮 提交于 2020-01-14 04:05:44

问题


I'm using the requests library in python 3 and despite my best efforts I can't get the following warning to disappear:

WARNING:requests.packages.urllib3.connectionpool:Connection pool is full, discarding connection: myorganization.zendesk.com

I'm using requests in a multithreaded environment to get and post json files concurrently to a single host, definitely no subdomains. In this current set up I'm using just 20 threads.

I attempted to use a Session in order to get requests to reuse connections and thus get rid of the problem, but it hasn't worked. This is the code in my class constructor:

self.session = requests.Session()
adapter = requests.adapters.HTTPAdapter(
    pool_connections=100, pool_maxsize=100)
self.session.mount('http://', adapter)
self.session.headers.update({'Connection':'Keep-Alive'})
self.session.auth = (self._user+"/token", self._token)

According to advice from here I shouldn't need to increase the pooled connections by that much considering the number of threads I'm using, but despite this I get this warning even when raising by 100.

This makes me think that connections are not being reused at all, or if they are, too many are being created for some reason. I've updated requests, so it is the most up to date version.

Does anyone have any ideas how I can get rid of this? I'm debugging some code and I think this is the blame for some requests not being made correctly.

Related:

Can I change the connection pool size for Python's "requests" module?


回答1:


Since zendesk communicates over https, you just need to mount the adapter to the https protocol, i.e.

self.session.mount('https://', adapter)


来源:https://stackoverflow.com/questions/29131873/python-requests-warning-urllib3-connectionpoolconnection-pool-is-full

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!