Concatenating or cascading multiple pretrained keras models

Question

I'm trying to build a concatenated or cascaded(actually don't even know if this is the correct definiton) set of models. For the simplicity my base models are looking like below.

                              ----Input----
                                    |
                                  L1-1
                                    |  
                                  L1-2
                                    |
                                  Dense
                                    |
                                 Softmax

I got 7 of these models trained with cross-validation and trying to wrap up them in a cascade fashion such as:

            -----------------------Input---------------------
            |       |       |       |       |       |       |       
          L1-1    L1-2    L1-3    L1-4     L1-5   L1-6    L1-7
            |       |       |       |       |       |       |
          L2-1    L2-2    L2-3    L2-4     L2-5   L2-6    L2-7
            |       |       |       |       |       |       |
            |_______|_______|_______|_______|_______|_______|
            |                  Concatenated                 |
            |___________________Dense Layer_________________|
                                    |
                                 SoftMax

Each one of Dense Layers got 512 neurons so in the end Concatenated Dense Layer would have a total of7*512=3584 neurons.

What I've done is:

• Trained all models and saved them in a list named as models[].
• Popped the bottom Softmax Layer in all models.

Then I try to concatenate them but got the error:

Layer merge was called with an input that isn't a symbolic tensor. 

What I'm gonna do after forming the cascade is freezing all the intermediate layers except Concatenated Dense Layer and tuning it up a little bit. But I'm stuck at as explained in all the details.

Answer 1

You need to use the functional API model for that. This kind of model works with tensors.

First you define a common input tensor:

inputTensor = Input(inputShape)

Then you call each model with this input to get the output tensors:

outputTensors = [m(inputTensor) for m in models]

Then you pass these tensors to the concatenate layer:

output = Concatenate()(outputTensors) 
output = Dense(...)(output)    
#you might want to use an Average layer instead of these two....

output = Activation('softmax')(output)

Finally, you define the complete model from start tensors to end tensors:

fullModel = Model(inputTensor,output)