Pickle load: ImportError: No module named doc2vec_ext

问题

This is the structure I'm dealing with:

src/
    processing/
        station_level/
            train_paragraph_vectors.py
    doc2vec_ext.py
    word_embeddings_station_level.py

I have trained and stored a model in word_embeddings_station_level.py like this:

from src.doc2vec_ext import WeightedDoc2Vec

# ...

model = WeightedDoc2Vec(
    # ...
)

train(model, vocab, station_sentences, num_epochs)

# Saving the model -> pickles it
model.save(open(model_file, "w"))

This is working fine so far. However, I want to load that model in train_paragraph_vectors.py like this:

import sys
from src import doc2vec_ext
sys.modules["doc2vec_ext"] = doc2vec_ext

if __name__ == "__main__":
# ...
    model = doc2vec_ext.WeightedDoc2Vec.load(station_level_sentence_vectors)

but I'm getting:

Traceback (most recent call last):
  File "E:/python/kaggle/seizure_prediction/src/processing/station_level/train_paragraph_vectors.py", line 57, in <module>
    model = doc2vec_ext.WeightedDoc2Vec.load(station_level_sentence_vectors)
  File "C:\Python27\lib\site-packages\gensim\models\word2vec.py", line 1684, in load
    model = super(Word2Vec, cls).load(*args, **kwargs)
  File "C:\Python27\lib\site-packages\gensim\utils.py", line 248, in load
    obj = unpickle(fname)
  File "C:\Python27\lib\site-packages\gensim\utils.py", line 911, in unpickle
    return _pickle.loads(f.read())
ImportError: No module named doc2vec_ext

doc2vec_ext.py

Here you can see, that I just inherit from the gensim.models.Doc2Vec class and do some stuff:

class WeightedDoc2Vec(Doc2Vec):

    def __init__(self, dm=1,window=5, f_size=0, size=100, min_count=1, negative=0, dbow_words=1, alpha=0.015, workers=8, seed=42, dm_weighted=False, dm_stacked=False):
        Doc2Vec.__init__(self,
            # Constructor arguments ..
            )

        # ...

I don't know what's the problem here. I've tried to do the sys.modules[] but it's still not working properly.

How can I load my stored model?

Important:

I noticed that I can't even load from the same module. If I try to load the model in the file where it was created (here word_embeddings_station_level.py) it's still not working giving me the same error.

回答1:

When saving and loading binary data like a pickle, you need to use binary mode, not text mode.

model.save(open(model_file, "wb"))  # 'b' for binary

来源：https://stackoverflow.com/questions/39944487/pickle-load-importerror-no-module-named-doc2vec-ext

标签

python

pickle

gensim