How to load a pickle object and resolve certain references

问题

Say we have an object we write to binary file using pickle. Let's say the object graph looks like this:

foo1
 +--->bar
 |     \--->context
 +--->baz
 |     +--->context
 |     \--->qux
 \--->context

Now the context objects are large datastructures and all instances of qux are the same. Therefore we decided to leave these contexxt objects out of the pickle process with:

def __getstate__(self):
    my_dict = dict(self.__dict__)
    my_dict['context'] = None  # We don't save the context
    return my_dict

for the classes to which foo, bar and baz belong (and in fact all other classes in our design).

When we load() the object back into memory, we want all context references to be set back to our given context. One can do this by walking over the object tree recursively, but we are wondering whether there is a more elegant solution in pickle that fills in the context values itself when loading the data back into memory?

回答1:

If the .context is of type Context, instead of setting context to None, you could adapt the code from the section Persistence of External Objects in the pickle documentation:

import pickle

class ContextAwarePickler(pickle.Pickler):
    def persistent_id(self, obj):
        # if this is a context, return the key
        if isinstance(obj, Context):
            return ("Context", context.key)

        # pickle as usual
        return None


class ContextAwareUnpickler(pickle.Unpickler):
    def recover_context(self, key_id):
        ...

    def persistent_load(self, pid):
        type_tag, key_id = pid
        if type_tag == "Context":
            return self.recover_context(key_id)
        else:
            raise pickle.UnpicklingError("unsupported persistent object")

来源：https://stackoverflow.com/questions/37026745/how-to-load-a-pickle-object-and-resolve-certain-references

标签

python

pickle