shouldn't model.trainable=False freeze weights under the model?

Question

I am trying to freeze the free trained VGG16\'s layers (\'conv_base\' below) and add new layers on top of them for feature extracting. I expect to get same prediction result

Answer 1

This is an interesting case. Why something like this happen is caused by the following thing:

You cannot freeze a whole model after compilation and it's not freezed if it's not compiled

If you set a flag model.trainable=False then while compiling keras sets all layers to be not trainable. If you set this flag after compilation - then it will not affect your model at all. The same - if you set this flag before compiling and then you'll reuse a part of a model for compiling another one - it will not affect your reused layers. So model.trainable=False works only when you'll apply it in a following order:

# model definition
model.trainable = False
model.compile()

In any other scenario it wouldn't work as expected.

Answer 2

You must freeze layers individually (before compilation):

for l in conv_base.layers: 
    l.trainable=False

And if this doesn't work, you should probably use the new sequential model to freeze the layers.

If you have models in models you should do this recursively:

def freezeLayer(layer):
    layer.trainable = False
    if hasattr(layer, 'layers'):
        for l in layer.layers:
            freezeLayer(l)

freezeLayer(model)

Answer 3

The top-rated answer does not work. As suggested by Keras official documentation (https://keras.io/getting-started/faq/), it should be performed per layer. Although there is a parameter "trainable" for a model, it is probably not implemented yet. The safest way is to do as follows:

for layer in model.layers:
    layer.trainable = False
model.compile()