How to connect Convlolutional layer with LSTM in tensorflow Keras

问题

I'm experimenting with architecture of neural network and I try to connect 2D convolution to LSTM cell in tensorflow Keras.

Here is my original model:

model = Sequential()

model.add(CuDNNLSTM(256, input_shape=(train_x.shape[1:]), return_sequences=True))
model.add(Dropout(0.2))
model.add(BatchNormalization())

model.add(Dense(64, activation='relu'))
model.add(Dropout(0.2))

model.add(Dense(4, activation='softmax'))

It works like magic.

train_x is 1209 sequences, each set has 23 numbers and sequence is 128 long. In other words - its shape is (1209, 128, 23). Input to the model is equal to train_x.shape[1:]=(128,23).

Now Iintend to add 256 Dense layer before LSTM cell, Reshape it to size 16x16, add a 2D convloution, Flatten it and connect it to LSTM cell. (and the same for layers following LSTM cell).

I started from:

model = Sequential()

model.add(Dense(256, input_shape=(train_x.shape[1:]), activation='relu'))
model.add(Dropout(0.2))

model.add(Reshape((16, 16), input_shape=(256,)))
model.add(Conv2D(16, (5,5), activation='relu', padding='same', input_shape=(16,16,1)))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())

model.add(CuDNNLSTM(256, return_sequences=True))
model.add(Dropout(0.1))
model.add(BatchNormalization())

model.add(Dense(64, activation='relu'))
model.add(Dropout(0.2))

model.add(Dense(4, activation='softmax'))

I have two errors:

Input 0 of layer conv2d is incompatible with the layer: expected ndim=4, found ndim=3. Full shape received: [None, 16, 16]

And when I remove convolution and leave only Reshape and Flatten layers:

tensorflow.python.framework.errors_impl.InvalidArgumentError: Input to reshape is a tensor with 4194304 values, but the requested shape has 32768
 [[{{node reshape/Reshape}} = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _class=["loc:@training/Adam/gradients/dropout/cond/Merge_grad/cond_grad"], _device="/job:localhost/replica:0/task:0/device:GPU:0"](dropout/cond/Merge, reshape/Reshape/shape)]]

Do you know how to deal with this?

来源：https://stackoverflow.com/questions/55431081/how-to-connect-convlolutional-layer-with-lstm-in-tensorflow-keras