dill

I have a pickling problem. I want to serialize a function in my main script, then load it and run it in another script. To demonstrate this, I've made 2 scripts: Attempt 1: The naive way: dill_pickle_script_1.py import pickle import time def my_func(a, b): time.sleep(0.1) # The purpose of this will become evident at the end return a+b if __name__ == '__main__': with open('testfile.pkl', 'wb') as f: pickle.dump(my_func, f) dill_pickle_script_2.py import pickle if __name__ == '__main__': with open('testfile.pkl') as f: func = pickle.load(f) assert func(1, 2)==3 Problem : when I run script 2, I

The question may seem a little basic, but wasn't able to find anything that I understood in the internet. How do I store something that I pickled with dill? I have come this far for saving my construct (pandas DataFrame, which also contains custom classes): import dill dill_file = open("data/2017-02-10_21:43_resultstatsDF", "wb") dill_file.write(dill.dumps(resultstatsDF)) dill_file.close() and for reading dill_file = open("data/2017-02-10_21:43_resultstatsDF", "rb") resultstatsDF_out = dill.load(dill_file.read()) dill_file.close() but I when reading I get the error TypeError: file must have

I have tried multiple approaches to pickle a python function with dependencies, following many recommendations on StackOverflow, (such as dill, cloudpickle, etc.) but all seem to run into a fundamental issue that I cannot figure out. I have a main module that tries to pickle a function from an imported module, sends it over ssh to be unpickled and executed at a remote machine. So main has: import dill (for example) import modulea serial=dill.dumps( modulea.func ) send (serial) On the remote machine: import dill receive serial funcremote = dill.loads( serial ) funcremote() If the functions

I need to serialise scikit-learn/statsmodels models such that all the dependencies (code + data) are packaged in an artefact and this artefact can be used to initialise the model and make predictions. Using the pickle module is not an option because this will only take care of the data dependency (the code will not be packaged). So, I have been conducting experiments with Dill . To make my question more precise, the following is an example where I build a model and persist it. from sklearn import datasets from sklearn import svm from sklearn.preprocessing import Normalizer import dill digits =

My program was killed while serializing data (a dict ) to disk with dill . I cannot open the partially-written file now. Is it possible to partially or fully recover the data? If so, how? Here's what I've tried: >>> dill.load(open(filename, 'rb')) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "lib/python3.4/site-packages/dill/dill.py", line 288, in load obj = pik.load() EOFError: Ran out of input >>> The file is not empty: >>> os.stat(filename).st_size 31110059 Note: all data in the dictionary was comprised of python built-in types. The pure-Python version of