Why is TF Keras inference way slower than Numpy operations?

Question

I\'m working on a reinforcement learning model implemented with Keras and Tensorflow. I have to do frequent calls to model.predict() on single inputs.

While testing infe

Answer 1

A little late, but maybe useful for someone:

Replace model.predict(X) with model.predict(X, batch_size=len(X))

That should do it.