Predicting next word using the language model tensorflow example

Question

The tensorflow tutorial on language model allows to compute the probability of sentences :

probabilities = tf.nn.softmax(logits)

in the co

Answer 1

You need to find the argmax of the probabilities, and translate the index back to a word by reversing the word_to_id map. To get this to work, you must save the probabilities in the model and then fetch them from the run_epoch function (you could also save just the argmax itself). Here's a snippet:

inverseDictionary = dict(zip(word_to_id.values(), word_to_id.keys()))

def run_epoch(...):
  decodedWordId = int(np.argmax(logits))
  print (" ".join([inverseDictionary[int(x1)] for x1 in np.nditer(x)])  
    + " got" + inverseDictionary[decodedWordId] + 
    + " expected:" + inverseDictionary[int(y)])

See full implementation here: https://github.com/nelken/tf