Using freebase vectors with gensim

问题

I am trying to use the freebase word embeddings released by Google, but I have a hard time getting the words from the freebase name.

model = gensim.models.Word2Vec.load_word2vec_format('freebase-vectors-skipgram1000.bin',binary=True)
model.vocab.keys()[:10]

Out[22]:
[u'/m/026tg5z',
 u'/m/018jz8',
 u'/m/04klsk',
 u'/m/08gd39',
 u'/m/0kt94',
 u'/m/05mtf0t',
 u'/m/05tjjb',
 u'/m/01m3vn',
 u'/m/0h7p35',
 u'/m/03ggvg3']

Does anyone know if it exist some kind of table to map the freebase representations into the words they represent ?

Regards,

Hedi

回答1:

Someone has actually done a nice thing for us all and mapped the IDs to the names in a pre-trained model. You can download that model here.

from gensim.models import Word2Vec
model = Word2Vec.load_word2vec_format('freebase-vectors-skipgram1000-en.bin.gz',
                                       binary=True)

Notice the extra -en before .bin. Then some sample vocab:

>>> list(model.vocab.keys())[:10] 
['/en/the_final_country', '/en/independent_curators_international', 
'/en/coney_reyes', '/en/scalr', '/en/everyman_palace_theatre', 
'/m/0g55w3s', '/en/waltershausen', '/en/river_frome_stroud', 
'/en/grzegorz_turnau']

回答2:

Those strings are Freebase identifiers, specifically MIDs, not names. You can look up their names using the Freebase MQLRead or Search APIs and they're also included in the Freebase data dumps.

The first ID in your example represents British film director Jack Gold. https://www.freebase.com/m/026tg5z

This API call will return JSON with his name:

https://www.googleapis.com/freebase/v1/mqlread?query=[{"id":"/m/026tg5z","name":null}]

来源：https://stackoverflow.com/questions/30480027/using-freebase-vectors-with-gensim

标签

python

freebase

gensim

word2vec