Why is TF Keras inference way slower than Numpy operations?

Question

I\'m working on a reinforcement learning model implemented with Keras and Tensorflow. I have to do frequent calls to model.predict() on single inputs.

While testing infe

Answer 1

Are you running your Keras model (with TensorFlow backend) in a loop? If so, Keras has a memory leak issue identified here: LINK

In this case you have to import the following:

import keras.backend.tensorflow_backend
import tensorflow as tf

from keras.backend import clear_session

Finally, you have to put the following at the end of every iteration of a loop after you're done doing your computations:

clear_session()
if keras.backend.tensorflow_backend._SESSION:
    tf.reset_default_graph()
    keras.backend.tensorflow_backend._SESSION.close()
    keras.backend.tensorflow_backend._SESSION = None

This should help you free up memory at the end of every loop and eventually, make the process faster. I hope this helps.