How to use pre-trained model as non trainable sub network in tensorflow?

Question

I\'d like to train a network that contains a sub network that I need to stay fix during the training. The basic idea is to prepend and append some layers the the pre-trained

Answer 1

Whether or not a "layer" is trained is determined by whether the variables used in that layer get updated with gradients. If you are using the Optimizer interface to optimize your network, then you can simply not pass the variables used in the layers that you want to keep fixed to the minimize function, i.e.,

opt.minimize(loss, <subset of variables you want to train>)

If you are using tf.gradients function directly, then remove the variables that you want to keep fixed from the second argument to tf.gradients.

Now, how you "branch directly" to a layer of a pre-trained network depends on how that network is implemented. I would simply locate the tf.Conv2D call to the 299x299 layer you are talking about, and pass as its input, the output of your new layer, and on the output side, locate the 79x79 layer, use its output as the input to your new layer.