TensorFlow: Adding Class to Pre-trained Inception Model & Outputting Full Image Hierarchy

Question

Two questions:

1) Does anyone know if I can add new image classes to the pre-trained Inception-v3 model? For example, I wanted to train TensorFlow on a multitude of

Answer 1

Yes you can, i recently did something very similar, in my case it was patogenic plant leaves vs healthy plant leaves. The v3 inception is already trained, what you will be doing is transfer learning. Transfer learning is a technique that shortcuts a lot of this work by taking a fully-trained model for a set of categories like ImageNet, and retrains from the existing weights for new classes.

Link : Image Retraining at tensorflow.org

Video Sources: YouTube video tutorial, Hvass Laboratories has some great video resource to rectify your questions.

Answer 2

1. Here is related question: How to get existing model to recognize additional classes?

Here is some explanations: https://github.com/tensorflow/models/issues/2510.

So it is possible somehow finetune model if having model checkpoint. Here is repo link with example of finetuning: https://github.com/tensorflow/models/tree/master/research/slim/

2. It is possible to get node name and human readable slug for class, so you can do manual linking of nodes by categories. But it is time consuming.

Answer 3

1) Output layer is softmax, which means it has predefined number of neurons, each one is defined for one specific class. Technically you could perform Network Surgery so that it has one more neuron in output layer which will represent your new class. But you will have to perform additional training of your network so that it updates all of its weights in order to account for new class. Bad news - it could take a while since update will affect whole network and the network is GIANT. Good news - such change in pretrained existing network will be faster than learning everything from scratch.

2) What makes you think that such hierarchy exists at all? You can't know anything about internal representation of data for sure. Of course, you can inspect activations of neurons in each of the functions and even visualize them... But you will have to try to understand what these activations mean on your own. And probably you won't find any hierarchy which you expect to see. So to sum up - understanding how ANN represents data internally is no easy task. Actually - extremely difficult one.

Suggested further reading: https://github.com/tensorflow/models/tree/master/inception

Pay attention to this part of the doc - it is strongly related to your #1