How to remove instancemethod objects, for the sake of pickle, without modifying the original class

问题

I want to persistantly hold on to an object from reverend.thomas.Bayes. Of course, if I try to pickle one of these classes directly, I get:

TypeError: can't pickle instancemethod objects

To work around this, I have tried declaring two functions:

import types
from itertools import chain
from copy import copy
from reverend.thomas import Bayes

def prepare_bayes_for_pickle(bayes_obj):
    dic = copy(bayes_obj.__dict__) #I also tried using deepcopy instead of copy
    for k in dic:
        if type(k) == types.MethodType:
            dic.pop(k)
    return dic

def reconstruct_bayes_from_pickle(bayes_dic):
    b = Bayes()
    # Merge b with bayes_dic, with bayes_dic taking precedence
    dic = dict(chain(bayes_dic, b))
    b.__dict__ = dic
    return b

Basically, I try copying the object's __dict__, and try to remove instancemethods by testing the type against types.MethodType.

I then would reconstruct the object by creating a new Bayes object and then merging it back together with bayes_dic (after it is UnPickled.)

But, I haven't gotten up to the second method yet, because I still can't pickle the object that is returned from prepare_bayes_for_pickle without getting the original error.

回答1:

The better solution would be for you to add a __getstate__ method onto the Bayes class (with accompanying __setstate__):

import types
from reverend.thomas import Bayes

def Bayes__getstate__(self):
    state = {}
    for attr, value in self.__dict__.iteritems():
        if not isinstance(value, types.MethodType):
            state[attr] = value
        elif attr == 'combiner' and value.__name__ == 'robinson':
            # by default, self.combiner is set to self.robinson
            state['combiner'] = None
    return state

def Bayes__setstate__(self, state):
    self.__dict__.update(state)
    # support the default combiner (an instance method):
    if 'combiner' in state and state['combiner'] is None:
        self.combiner = self.robinson

Bayes.__getstate__ = Bayes__getstate__
Bayes.__setstate__ = Bayes__setstate__

Now the Bayes class can always be pickled and unpickled without additional processing.

I do see that the class has a self.cache = {} mapping; perhaps that should be excluded when pickling? Ignore it in __getstate__ and call self.buildCache() in __setstate__ if that is the case.

回答2:

k is a key i.e. the attribute/method name. You need to test the attribute itself:

    if type(dic[k]) == types.MethodType:
            ^~~~~~ here

I'd prefer using a comprehension; you should also be using isinstance:

dic = dict((k, v) for k, v in bayes_obj.__dict__
           if not isinstance(v, types.MethodType))

回答3:

This sounds like getting a square peg to fit a round hole. How about using pickle to pickle the arguments, and unpickle to reconstruct the reverand.Thomas.Bayes object?

>>> from collections import namedtuple
>>> ArgList = namedtuple('your', 'arguments', 'for', 'the', 'reverand')
>>> def pickle_rtb(n):
...     return pickle.dumps(ArgList(*n.args))
... 
>>> def unpickle_rtb(s):
...     return reverand.Thomas.Bayes(*pickle.loads(s))
... 
>>> s = pickle_rtb(reverand.Thomas.Bayes(1, 2, 3, 4, 5)) # note arguments are a guess
>>> rtb = unpickle_norm(s)

Inspired by this SO question.

来源：https://stackoverflow.com/questions/15499170/how-to-remove-instancemethod-objects-for-the-sake-of-pickle-without-modifying

标签

python

python-2.7

pickle