How does word2vec give one hot word vector from the embedding vector?

问题

I understand how word2vec works.

I want to use word2vec(skip-gram) as input for RNN. Input is embedding word vector. Output is also embedding word vector generated by RNN.

Here’s question! How can I convert the output vector to one hot word vector? I need inverse matrix of embeddings but I don’t have!

回答1:

The output of an RNN is not an embedding. We convert the output from the last layer in an RNN cell into a vector of vocabulary_size by multiplying with an appropriate matrix.

Take a look at the PTB Language Model example to get a better idea. Specifically look at lines 133-136:

softmax_w = tf.get_variable("softmax_w", [size, vocab_size], dtype=data_type())
softmax_b = tf.get_variable("softmax_b", [vocab_size], dtype=data_type())
logits = tf.matmul(output, softmax_w) + softmax_b

The above operation will give you logits. This logits is a probability distribution over your vocabulary. numpy.random.choice might help you to use these logits to make a prediction.

来源：https://stackoverflow.com/questions/38738821/how-does-word2vec-give-one-hot-word-vector-from-the-embedding-vector