Is the Session object from Python's Requests library thread safe?

Question

Python's popular Requests library is said to be thread-safe on its home page, but no further details are given. If I call requests.session(), can I then safely pass this object to multiple threads like so:

session = requests.session()
for i in xrange(thread_count):
    threading.Thread(
        target=target,
        args=(session,),
        kwargs={}
    )

and make requests using the same connection pool in multiple threads?

If so, is this the recommended approach, or should each thread be given its own connection pool? (Assuming the total size of all the individual connection pools summed to the size of what would be one big connection pool, like the one above.) What are the pros and cons of each approach?

Answer 1

After reviewing the source of requests.session, I'm going to say the session object might be thread-safe, depending on the implementation of CookieJar being used.

Session.prepare_request reads from self.cookies, and Session.send calls extract_cookies_to_jar(self.cookies, ...), and that calls jar.extract_cookies(...) (jar being self.cookies in this case).

The source for Python 2.7's cookielib acquires a lock (threading.RLock) while it updates the jar, so it appears to be thread-safe. On the other hand, the documentation for cookielib says nothing about thread-safety, so maybe this feature should not be depended on?

UPDATE

If your threads are mutating any attributes of the session object such as headers, proxies, stream, etc. or calling the mount method or using the session with the with statement, etc. then it is not thread-safe.

Answer 2

https://github.com/kennethreitz/requests/issues/1871 implies that Session is not thread-safe, and that at least one maintainer recommends one Session per thread.

I just opened https://github.com/kennethreitz/requests/issues/2766 to clarify the documentation.