How to serialize sympy lambdified function?

Question

The title says it all. Is there any way to serialize a function generated by sympy.lambdify?:

import sympy as sym
import pickle
import dill
a, b = sym.symbols("a, b")
expr = sym.sin(a) + sym.cos(b)
lambdified_expr = sym.lambdify((a, b), expr, modules="numpy")
pickle.dumps(lambdified_expr) # won't work
dill.dumps(lambdified_expr) # won't work either

... The reason I want to do this is because my code generates so many lambdified functions but I found it takes too long every time.

Answer 1

You actually can use dill to pickle it. The most recent versions of dill (e.g. on github) has "settings" that allow variants of how the pickle is constructed on dump. Yes, the default settings for dill fail on this object, but not if you use the setting that recursively traces global references (i.e. recurse = True). This setting is similar to what cloudpickle gives you by default.

>>> import sympy as sym
>>> import pickle
>>> import dill
>>> a, b = symbols("a, b")
>>> a, b = sym.symbols("a, b")
>>> expr = sym.sin(a) + sym.cos(b)
>>> lambdified_expr = sym.lambdify((a, b), expr, modules="numpy")
>>> 
>>> dill.settings
{'recurse': False, 'byref': False, 'protocol': 2, 'fmode': 0}
>>> dill.settings['recurse'] = True
>>> dill.dumps(lambdified_expr)
'\x80\x02cdill.dill\n_create_function\nq\x00(cdill.dill\n_unmarshal\nq\x01U\x83c\x02\x00\x00\x00\x02\x00\x00\x00\x03\x00\x00\x00C \x00\x00s\x14\x00\x00\x00t\x00\x00|\x00\x00\x83\x01\x00t\x01\x00|\x01\x00\x83\x01\x00\x17S(\x01\x00\x00\x00N(\x02\x00\x00\x00t\x03\x00\x00\x00sint\x03\x00\x00\x00cos(\x02\x00\x00\x00t\x01\x00\x00\x00at\x01\x00\x00\x00b(\x00\x00\x00\x00(\x00\x00\x00\x00s\x08\x00\x00\x00<string>t\x08\x00\x00\x00<lambda>\x01\x00\x00\x00s\x00\x00\x00\x00q\x02\x85q\x03Rq\x04}q\x05(U\x03cosq\x06cnumpy.core.umath\ncos\nq\x07U\x03sinq\x08cnumpy.core.umath\nsin\nq\tuU\x08<lambda>q\nNN}q\x0btq\x0cRq\r.'

P.S. I'm the dill author, so I'd know.

Answer 2

Indeed - pickle, cPickle, and even dill fail on this example with default settings.

But cloudpickle does not fail!

pip install cloudpickle

or

https://github.com/cloudpipe/cloudpickle

import sympy as sym
from cloudpickle import dumps, loads
a, b = sym.symbols("a, b")
expr = sym.sin(a) + sym.cos(b)
lambdified_expr = sym.lambdify((a, b), expr, modules="numpy")
var=dumps(lambdified_expr)
a1=lambdified_expr(10,10)
del lambdified_expr
lambdified_expr=loads(var)
a2=lambdified_expr(10,10)
a1==a2  # True