pickle

问题 Beginner here, want to read in data with the file ending p. My code looks like this : import pickle training_file = "/home/sk/CarND-Traffic-Sign-Classifier-Project/train.p" testing_file = "/home/sk/CarND-Traffic-Sign-Classifier-Project/test.p" with open(training_file, mode='rb') as f: train = pickle.load(f) with open(testing_file, mode='rb') as f: test = pickle.load(f) I get the following error: ValueError: unsupported pickle protocol: 3 Can someone point out how i can fix it, either changing

问题 See updates at the bottom -- A similar question was asked here, but never resolved: pickling and unpickling user-defined class I'm working on a project which necessitates pickling user defined classes, and sending them to a remote server where they are unpickled and called. We use the Dill library to accomplish this, and have had a lot of success. Unfortunately, I've run into an issue I'm having a hard time debugging. I create and pickle a class as follows: import dill, base64 import time,

问题 I'm using Python 3.6 and Spyder (Anaconda). I have tried many things but nothing worked out. I don't know why this error is coming always to me while loading with pickle. filename = "allfeatures.txt" allfeatures = open(filename, 'r').read() with open(filename) as f: allfeatures = list(f) allconcat = np.vstack(list(allfeatures.values())) TypeError Traceback (most recent call last) AttributeError: 'list' object has no attribute 'values' 回答1: You need to open your file as a binary file: pickle

问题 I'm using Python 3.6 and Spyder (Anaconda). I have tried many things but nothing worked out. I don't know why this error is coming always to me while loading with pickle. filename = "allfeatures.txt" allfeatures = open(filename, 'r').read() with open(filename) as f: allfeatures = list(f) allconcat = np.vstack(list(allfeatures.values())) TypeError Traceback (most recent call last) AttributeError: 'list' object has no attribute 'values' 回答1: You need to open your file as a binary file: pickle

问题 We have trained an Extra Tree model for some regression task. Our model consists of 3 extra trees, each having 200 trees of depth 30. On top of the 3 extra trees, we use a ridge regression. We train our model for several hours and pickle the trained model (the entire class object), for later use. However, the size of saved trained model is too big, about 140 GB! Is there a way to reduce the size of the saved model? are there any configuration in pickle that could be helpful, or any

问题 I have huge array objects that are pickled with the python pickler. I am trying to unpickle them and reading out the data in a for loop. Every time I am done reading and assesing, I delete all the references to those objects. After deletion, I even call gc.collect() along with time.sleep() to see if the heap memory reduces. The heap memory doesn't reduce pointing to the fact that, the data is still referenced somewhere within the pickle loading. After 15 datafiles(I got 250+ files to process,

问题 I assumed that the multiprocessing package used pickle to send things between processes. However, pickle pays attention to the __getstate__ and __setstate__ methods of an object. Multiprocessing seems to ignore them. Is this correct? Am I confused? To replicate, install docker, and type into command line $ docker run python:3.4 python -c "import pickle import multiprocessing import os class Tricky: def __init__(self,x): self.data=x def __setstate__(self,d): self.data=10 def __getstate__(self)

问题 I have dill installed in my python 2.7 but when I try to unpickle my model it says "No module named dill". The pickled file contains pandas series. EDIT : Here's the snapshot of the traceback on ElasticBeanstalk environment File "/opt/python/current/app/app/models/classification.py", line 663, in __init__ self.lookupdict = pickle.load(open(<filepath>)) File "/usr/lib64/python2.7/pickle.py", line 1384, in load return Unpickler(file).load() File "/usr/lib64/python2.7/pickle.py", line 864, in

问题 I have dill installed in my python 2.7 but when I try to unpickle my model it says "No module named dill". The pickled file contains pandas series. EDIT : Here's the snapshot of the traceback on ElasticBeanstalk environment File "/opt/python/current/app/app/models/classification.py", line 663, in __init__ self.lookupdict = pickle.load(open(<filepath>)) File "/usr/lib64/python2.7/pickle.py", line 1384, in load return Unpickler(file).load() File "/usr/lib64/python2.7/pickle.py", line 864, in

问题 I need to execute the code below (simplified version of my real code base in Python 3.5): import multiprocessing def forever(do_something=None): while True: do_something() p = multiprocessing.Process(target=forever, args=(lambda: print("do something"),)) p.start() In order to create the new process Python need to pickle the function and the lambda passed as target. Unofrtunately pickle cannot serialize lambdas and the output is like this: _pickle.PicklingError: Can't pickle <function <lambda>