Pickle can't store an object in django locmem cache during tests?

Question

Something that\'s puzzling me a bit...

>>> from django.core.cache import get_cache
>>>
>>> cache = get_cache(\'django.core.cache.backe         


        

Answer 1

The problem is that pickle serializes classes by reference, so could you not just use a better serializer that pickles the class by serializing the class definitions instead of by reference? Then you'd pickle a mock object, which would then pickle the class source code, and then you'd be able to pass that to the django cache. I'm the author of dill, which is a better serializer… and also the author of klepto, which is a caching package… and this is exactly what I do to store any object in a SQL table, on disk, or in an in-memory cache.

Essentially (not trying this, but guessing it works based on experience with my own caching package), it should work like this:

>>> from django.core.cache import get_cache
>>> import dill
>>>
>>> cache = get_cache('django.core.cache.backends.locmem.LocMemCache')
>>>
>>> # Set the 'content' cache key to a string
>>> cache.set('content', dill.dumps('a string'))
>>> dill.loads(cache.get('content'))
'a string'
>>>
>>> class TestObj(object):
...     pass
>>>
>>> a = TestObj()
>>> cache.set('content', dill.dumps(a))
>>>
>>> dill.loads(cache.get('content'))
<__main__.TestObj object at 0x10235e510>
>>>
>>> # this is pickling classes w/o using a reference
>>> dill.dumps(a)
'\x80\x02cdill.dill\n_create_type\nq\x00(cdill.dill\n_load_type\nq\x01U\x08TypeTypeq\x02\x85q\x03Rq\x04U\x07TestObjq\x05h\x01U\nObjectTypeq\x06\x85q\x07Rq\x08\x85q\t}q\n(U\r__slotnames__q\x0b]q\x0cU\n__module__q\rU\x08__main__q\x0eU\x07__doc__q\x0fNutq\x10Rq\x11)\x81q\x12}q\x13b.'
>>> # and here's using a reference, which is exactly how pickle does it
>>> dill.dumps(a, byref=True)
'\x80\x02c__main__\nTestObj\nq\x00)\x81q\x01}q\x02b.'

If you want to try it yourself, get dill (and klepto) here: https://github.com/uqfoundation