How to save requests (python) cookies to a file?

Question

How to use the library requests (in python) after a request

#!/usr/bin/env python
# -*- coding: utf-8 -*-
import requests
bot = requests.session         


        

Answer 1

After a call such as r = requests.get(), r.cookies will return a RequestsCookieJar which you can directly pickle, i.e.

import pickle
def save_cookies(requests_cookiejar, filename):
    with open(filename, 'wb') as f:
        pickle.dump(requests_cookiejar, f)

def load_cookies(filename):
    with open(filename, 'rb') as f:
        return pickle.load(f)

#save cookies
r = requests.get(url)
save_cookies(r.cookies, filename)

#load cookies and do a request
requests.get(url, cookies=load_cookies(filename))

If you want to save your cookies in human-readable format, you have to do some work to extract the RequestsCookieJar to a LWPCookieJar.

import cookielib
def save_cookies_lwp(cookiejar, filename):
    lwp_cookiejar = cookielib.LWPCookieJar()
    for c in cookiejar:
        args = dict(vars(c).items())
        args['rest'] = args['_rest']
        del args['_rest']
        c = cookielib.Cookie(**args)
        lwp_cookiejar.set_cookie(c)
    lwp_cookiejar.save(filename, ignore_discard=True)

def load_cookies_from_lwp(filename):
    lwp_cookiejar = cookielib.LWPCookieJar()
    lwp_cookiejar.load(filename, ignore_discard=True)
    return lwp_cookiejar

#save human-readable
r = requests.get(url)
save_cookies_lwp(r.cookies, filename)

#you can pass a LWPCookieJar directly to requests
requests.get(url, cookies=load_cookies_from_lwp(filename))

Answer 2

You can pickle the cookies object directly:

cookies = pickle.dumps(session.cookies)

The dict representation misses a lot of informations: expiration, domain, path...

It depends on the usage you intend to do with the cookies, but if you don't have informations about the expiration, for example, you should implement the logic to track expiration by hand.

Pickling the library returned object lets you easily reconstruct the state, then you can relay on the library implementation.

Obviously, this way, the consumer of the pickled object needs to use the same library

Answer 3

I offer a way by json:

to save cookie -

import json
with open('cookie.txt', 'w') as f:
    json.dump(requests.utils.dict_from_cookiejar(bot.cookies), f)

and to load cookie -

import json
session = requests.session()  # or an existing session

with open('cookie.txt', 'r') as f:
    cookies = requests.utils.cookiejar_from_dict(json.load(f))
    session.cookies.update(cookies)

Answer 4

Simple way to convert cookies into list of dicts and save to json or db. This is methods of class which have session attribute.

def dump_cookies(self):
    cookies = []
    for c in self.session.cookies:
        cookies.append({
            "name": c.name,
            "value": c.value,
            "domain": c.domain,
            "path": c.path,
            "expires": c.expires
        })
    return cookies

def load_cookies(self, cookies):
    for c in cookies:
        self.session.cookies.set(**c)

All we need is five parameters such as: name, value, domain, path, expires