Transfer learning from the pre-trained NASnet network. How to know the number of layers to freeze?

Question

In order to train a model for image classification (using Keras or Tensorflow) I want to retrain a certain number of layers of the NASNetMobile, using my own dataset of images.

In this paper: https://arxiv.org/pdf/1707.07012.pdf (section A.7) we can read: "Additionally, all models use an auxiliary classifier located at 2/3 of the way up the network".

Here the layers of the NasNetMobile that I want to do the transfer learning from: https://gist.github.com/didacroyo/a451c890b1f02822c7dd67c6f270f1d6

Then, based on the previous, should I freeze the bottom 1/3 of layers? (this is the first 250 layers)

Answer 1

Considering where the Aux branch begins, I'd try freezing the layers before activation_166. Something like this:

model = NASNetLarge((img_rows, img_cols, img_channels),dropout=0.5, use_auxiliary_branch=True, include_top=True, weights=None, classes=nb_classes)

model.load_weights('weights/NASNet-large.h5', by_name=True, skip_mismatch=True)

# Freeze original layers
model.trainable = True    
set_trainable = False
for layer in model.layers:
  if layer.name == 'activation_166':
    set_trainable = True
  if set_trainable:
    layer.trainable = True
  else:
    layer.trainable = False
  print("layer {} is {}".format(layer.name, '+++trainable' if layer.trainable else '---frozen'))