dill

问题 I have a large object in my Python3 code which, when tried to be pickled with the pickle module throws the following error: TypeError: cannot serialize '_io.BufferedReader' object However, dill.dump() and dill.load() are able to save and restore the object seamlessly. What causes the trouble for the pickle module? Now that dill saves and reconstructs the object without any error, is there any way to verify if the pickling and unpickling with dill went well? How's it possible that pickle fails

问题 I have a large object in my Python3 code which, when tried to be pickled with the pickle module throws the following error: TypeError: cannot serialize '_io.BufferedReader' object However, dill.dump() and dill.load() are able to save and restore the object seamlessly. What causes the trouble for the pickle module? Now that dill saves and reconstructs the object without any error, is there any way to verify if the pickling and unpickling with dill went well? How's it possible that pickle fails

问题 If a dill file is to large to RAM, is it possible to load it in an alternative method. For example python3 is throwing Memory Errors when I load a serialized object of about 1.2 GB. file = open('SerializedData.pkl', 'rb') data = dill.load(file) This will not pass because the RAM is too small. However, is it possible to load it in a different way so that I extract the data without overloading the RAM? 来源： https://stackoverflow.com/questions/32465186/if-dill-file-is-too-large-for-ram-is-there

问题 I need to serialise scikit-learn/statsmodels models such that all the dependencies (code + data) are packaged in an artefact and this artefact can be used to initialise the model and make predictions. Using the pickle module is not an option because this will only take care of the data dependency (the code will not be packaged). So, I have been conducting experiments with Dill. To make my question more precise, the following is an example where I build a model and persist it. from sklearn

问题 In the example below, I have placed the class Foo inside its own module foo . Why is the external class dumped by ref? The instance ff is not being dumped with its source code. I am using Python 3.4.3 and dill-0.2.4. import dill import foo class Foo: y = 1 def bar( self, x ): return x + y f = Foo() ff = foo.Foo() print( dill.dumps( f, byref=False, recurse=True ) ) print( '\n' ) print( dill.dumps( ff, byref=False, recurse=True ) ) Well, the code above is actually wrong (should be Foo.y ,

问题 I have tried multiple approaches to pickle a python function with dependencies, following many recommendations on StackOverflow, (such as dill, cloudpickle, etc.) but all seem to run into a fundamental issue that I cannot figure out. I have a main module that tries to pickle a function from an imported module, sends it over ssh to be unpickled and executed at a remote machine. So main has: import dill (for example) import modulea serial=dill.dumps( modulea.func ) send (serial) On the remote

问题 Dill.detect.children requires two arguments; obj and objtype . Inspecting an audiofile object I can call: dill.detect.children(audiofile, object) dill.detect.children(audiofile, dict) dill.detect.children(audiofile, list) Which return without error. But how about looking for instance methods? type(audiofile.save) returns instancemethod Tried dill.detect.children(audiofile, instancemethod) which returns NameError: name 'instancemethod' is not defined Tried dill.detect.children(audiofile,

问题 I am using dill (advanced version of pickle) right now. I want to serialize my object, but I get this error: /usr/lib/python2.7/pickle.pyc in memoize(self, obj) 242 if self.fast: 243 return --> 244 assert id(obj) not in self.memo 245 memo_len = len(self.memo) 246 self.write(self.put(memo_len)) Can someone tell me the possibility that made this error or how can I solved this? 回答1: Without you posting a reduced version of your code, it's hard to help. However, dill has some builtin detection

问题 Trying to render a large and (possibly very) unpicklable object to a file for later use. No complaints on the dill.dump(file) side: In [1]: import echonest.remix.audio as audio In [2]: import dill In [3]: audiofile = audio.LocalAudioFile("/Users/path/Track01.mp3") en-ffmpeg -i "/Users/path/audio/Track01.mp3" -y -ac 2 -ar 44100 "/var/folders/X2/X2KGhecyG0aQhzRDohJqtU+++TI/-Tmp-/tmpWbonbH.wav" Computed MD5 of file is b3820c166a014b7fb8abe15f42bbf26e Probing for existing analysis In [4]: with

问题 dill is a great tool for pickling most the Python objects, I use it in IPython parallel to serialize calculations. One issue I keep getting into is around dill-ing class definitions. One of the errors I get is explained below. While trying to serialize class definitions, I keep getting AssertionError from dill . I wonder why one of these works and the other fails: class MyClassEmpty(object): pass class MyClassInit(object): def __init__(self): super(MyClassInit).__init__() dill.dumps