Estimate required resources to serve Keras model

问题

I have a Keras model (.hdf5) that I would like to deploy in the cloud for prediction. I now wish to estimate how much resources I need for this (CPU, GPU, RAM, ...).

Does anyone have a suggestion for functions / rules of thumb that could help with this? I was unable to find anything useful. Thanks in advance!

回答1:

I think the most realistic estimation would be to run the model and see how much resources does it take. top or htop will show you the CPU and RAM load, but in case of GPU memory it is a bit more complicated, since TensorFlow (most popular option for the Keras backend) reserves all the available memory for performance reasons.

You have to tell TensorFlow not to take all available memory but allocate it on demand. Here is how to perform this in Keras:

import tensorflow as tf
import keras.backend as K
config = tf.ConfigProto()
config.gpu_options.per_process_gpu_memory_fraction=0.2  # Initially allocate only 20% of memory
config.gpu_options.allow_growth = True  # dynamically grow the memory used on the GPU
config.log_device_placement = True  # to log device placement (on which device the operation ran)
                                    # (nothing gets printed in Jupyter, only if you run it standalone)
sess = tf.Session(config=config)
K.set_session(sess)  # set this TensorFlow session as the default session for Keras

https://github.com/keras-team/keras/issues/4161#issuecomment-366031228

Then, run watch nvidia-smi and see how much memory will be taken.

来源：https://stackoverflow.com/questions/54276040/estimate-required-resources-to-serve-keras-model