pickle

问题 I want my class to implement Save and Load functions which simply do a pickle of the class. But apparently you cannot use 'self' in the fashion below. How can you do this? self = cPickle.load(f) cPickle.dump(self,f,2) 回答1: This is what I ended up doing. Updating the __dict__ means we keep any new member variables I add to the class and just update the ones that were there when the object was last pickle'd. It seems the simplest while maintaining the saving and loading code inside the class

问题 I'm applying some parallelization to my code, in which I use classes. I knew that is not possible to pick a class method without any other approach different of what Python provides. I found a solution here. In my code, I have to parts that should be parallelized, both using class. Here, I'm posting a very simple code just representing the structure of mine (is the same, but I deleted the methods content, which was a lot of math calculus, insignificant for the output that I'm getting). The

问题 I've spent the last couple of days getting to grips with the basics of lxml; in particular using lxml.html to parse websites and create an ElementTree of the content. Ideally, I want to save the returned ElementTree so that I can load it up and experiment with it, without having to parse the website every time I modify my script. I assumed that pickling would be the way to go, however I'm now beginning to wonder. Although I am able to retrieve an ElementTree object after pickling... type

问题 In the answer to Python pickle: dealing with updated class definitions, the author of the dill package writes: "Ok, I have added this feature to dill in the latest revision on github. Implemented with far less trickery than I thought... just serialize the class definition with the pickle, and voila." Having installed dill and tinkered with it, it's not obvious to me how to actually use this functionality in dill . Could someone provide an explicit example? I would like to pickle the class

问题 I am running code that creates large objects, containing multiple user-defined classes, which I must then serialize for later use. From what I can tell, only pickling is versatile enough for my requirements. I've been using cPickle to store them but the objects it generates are approximately 40G in size, from code that runs in 500 mb of memory. Speed of serialization isn't an issue, but size of the object is. Are there any tips or alternate processes I can use to make the pickles smaller? 回答1

问题 I am using pickle module in Python and trying different file IO modes: # works on windows.. "rb" with open(pickle_f, 'rb') as fhand: obj = pickle.load(fhand) # works on linux.. "r" with open(pickle_f, 'r') as fhand: obj = pickle.load(fhand) # works on both "r+b" with open(pickle_f, 'r+b') as fhand: obj = pickle.load(fhand) I never read about "r+b" mode anywhere, but found mentioning about it in the documentation. I am getting EOFError on Linux if I use "rb" mode and on Windows if "r" is used.

问题 When I am trying to load something I dumped using cPickle, I get the error message: ValueError: insecure string pickle Both the dumping and loading work are done on the same computer, thus same OS: Ubuntu 8.04. How could I solve this problem? 回答1: "are much more likely than a never-observed bug in Python itself in a functionality that's used billions of times a day all over the world": it always amazes me how cross people get in these forums. One easy way to get this problem is by forgetting

问题 I've created a subclass of numpy ndarray following the numpy documentation. In particular, I have added a custom attribute by modifying the code provided. I'm manipulating instances of this class within a parallel loop, using Python multiprocessing . As I understand it, the way that the scope is essentially 'copied' to multiple threads is using pickle . The problem I am now coming up against relates to the way that numpy arrays are pickled. I can't find any comprehensive documentation about

问题 I am trying to pickle a namedtuple : from collections import namedtuple import cPickle class Foo: Bar = namedtuple('Bar', ['x', 'y']) def baz(self): s = set() s.add(Foo.Bar(x=2, y=3)) print cPickle.dumps(s) if __name__ == '__main__': f = Foo() f.baz() This produces the following output: Traceback (most recent call last): File "scratch.py", line 15, in <module> f.baz() File "scratch.py", line 11, in baz print cPickle.dumps(s) cPickle.PicklingError: Can't pickle <class '__main__.Bar'>:

问题 I am trying to pickle a namedtuple : from collections import namedtuple import cPickle class Foo: Bar = namedtuple('Bar', ['x', 'y']) def baz(self): s = set() s.add(Foo.Bar(x=2, y=3)) print cPickle.dumps(s) if __name__ == '__main__': f = Foo() f.baz() This produces the following output: Traceback (most recent call last): File "scratch.py", line 15, in <module> f.baz() File "scratch.py", line 11, in baz print cPickle.dumps(s) cPickle.PicklingError: Can't pickle <class '__main__.Bar'>: