What does `training=True` mean when calling a TensorFlow Keras model?

Question

In TensorFlow\'s offcial documentations, they always pass training=True when calling a Keras model in a training loop, for example, logits = mnist_model(i

Answer 1

Some neural network layers behave differently during training and inference, for example Dropout and BatchNormalization layers. For example

• During training, dropout will randomly drop out units and correspondingly scale up activations of the remaining units.
• During inference, it does nothing (since you usually don't want the randomness of dropping out units here).

The training argument lets the layer know which of the two "paths" it should take. If you set this incorrectly, your network might not behave as expected.