Merge layers (concatenate) in keras

问题

I am trying to implement this paper (the model architecture is given below) and have two models- coarse_model and fine_model which need to be concatenated at the second step of the fine model. However, I am getting an error when I trying to concatenate using the last axis.

---

from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D
from keras.layers import Activation, Dropout, Flatten, Dense, Merge
from keras.layers.core import Reshape
from keras.layers.merge import Concatenate
from keras import backend as K


# dimensions of our images
#img_width, img_height = 320, 240

img_width, img_height = 304,228


if K.image_data_format() == 'channels_first':
    input_shape = (3, img_width, img_height)
else:
    input_shape = (img_width, img_height, 3)

# coarse model
coarse_model = Sequential()
# coarse layer 1
coarse_model.add(Conv2D(96,(11,11),strides=(4,4),input_shape=input_shape,activation='relu'))
coarse_model.add(MaxPooling2D(pool_size=(2, 2)))
# coarse layer 2
coarse_model.add(Conv2D(256,(5,5),activation='relu',padding='same'))
coarse_model.add(MaxPooling2D(pool_size=(2, 2)))
# coarse layer 3
coarse_model.add(Conv2D(384,(3,3),activation='relu',padding='same'))
# coarse layer 4
coarse_model.add(Conv2D(384,(3,3),activation='relu',padding='same'))
# coarse layer 5
coarse_model.add(Conv2D(256,(3,3),activation='relu',padding='same'))
coarse_model.add(Flatten())
# coarse layer 6
coarse_model.add(Dense(4096,activation='relu'))
# coarse layer 7
coarse_model.add(Dense(4070,activation='linear'))

# fine model
fine_model = Sequential()
fine_model.add(Conv2D(63,(9,9),strides=(2,2),input_shape=input_shape,activation='relu'))
fine_model.add(MaxPooling2D(pool_size=(2, 2)))

# reshape coarse model to shape of fine model
shape = fine_model.layers[1].output_shape
shape_subset = (shape[1],shape[2])


coarse_model.add(Reshape(shape_subset))
model = Sequential()
model.add(Merge([coarse_model.layers[10],fine_model.layers[1]],mode='concat',concat_axis=3))

---

The error given on the last line is: *** ValueError: "concat" mode can only merge layers with matching output shapes except for the concat axis. Layer shapes: [(None, 74, 55), (None, 74, 55, 63)]

回答1:

To answer my own question, changing the shape to

shape_subset = (shape[1],shape[2],1)

and

model.add(Merge([coarse_model.layers[10],fine_model.layers[1]],mode='concat',concat_axis=-1))

makes the code work.

来源：https://stackoverflow.com/questions/44787050/merge-layers-concatenate-in-keras