Keras - Making two predictions from one neural network

Question

I'm trying to combine two outputs that are produced by the same network that makes predictions on a 4 class task and a 10 class task. Then I look to combine these outputs to give a length 14 array which I use as my end target.

While this seems to work actively the predictions are always for one class so it produces a probability dist which is only concerned with selecting 1 out of the 14 options instead of 2. What I actually need it to do is to provide 2 predictions, one for each class. I want this all to be produced by the same model.

input = Input(shape=(100, 100), name='input')
lstm = LSTM(128, input_shape=(100, 100)))(input)

output1 = Dense(len(4), activation='softmax', name='output1')(lstm)
output2 = Dense(len(10), activation='softmax', name='output2')(lstm)

output3 = concatenate([output1, output2])

model = Model(inputs=[input], outputs=[output3])

My issue here is determining an appropriate loss function and method of prediction? For prediction I can simply grab the output of each layer after the softmax however I'm unsure how to set the loss function for each of these things to be trained.

Any ideas?

Thanks a lot

Answer 1

You don't need to concatenate the outputs, your model can have two outputs:

input = Input(shape=(100, 100), name='input')
lstm = LSTM(128, input_shape=(100, 100)))(input)

output1 = Dense(len(4), activation='softmax', name='output1')(lstm)
output2 = Dense(len(10), activation='softmax', name='output2')(lstm)

model = Model(inputs=[input], outputs=[output1, output2])

Then to train this model, you typically use two losses that are weighted to produce a single loss:

model.compile(optimizer='sgd', loss=['categorical_crossentropy', 
              'categorical_crossentropy'], loss_weights=[0.2, 0.8])

Just make sure to format your data right, as now each input sample corresponds to two output labeled samples. For more information check the Functional API Guide.