Keras - How to remove useless dimension without hurting the computation graph?

问题

While generating a deep learning model, I used K.squeeze function to squeeze useless dimension when the first two dimensions were None shape.

import keras.backend as K
>>> K.int_shape(user_input_for_TD)
(None, None, 1, 32)
>>> K.int_shape(K.squeeze(user_input_for_TD, axis=-2))
(None, None, 32)

However, this gives below error, It seems like K.squeeze function hurts the computation graph, is there any solution to escape from this issue? Maybe that function does not support calculating gradients, which isn't differentiable.

File "/home/sundong/anaconda3/envs/py36/lib/python3.6/site-packages/keras/engine/network.py", line 1325, in build_map
    node = layer._inbound_nodes[node_index]
AttributeError: 'NoneType' object has no attribute '_inbound_nodes'

Below code block is the whole code block which causes that error.

user_embedding_layer = Embedding(
            input_dim=len(self.data.visit_embedding),
            output_dim=32,
            weights=[np.array(list(self.data.visit_embedding.values()))],
            input_length=1,
            trainable=False)
...
all_areas_lstm = LSTM(1024, return_sequences=True)(all_areas_rslt)   # (None, None, 1024)
user_input_for_TD = Lambda(lambda x: x[:, :, 0:1])(multiple_inputs)  # (None, None, 1) 
user_input_for_TD = TimeDistributed(user_embedding_layer)(user_input_for_TD) # (None, None, 1, 32) 
user_input_for_TD = K.squeeze(user_input_for_TD, axis=-2) # (None, None, 32) 
aggre_threeway_inputs = Concatenate()([user_input_for_TD, all_areas_lstm]) # should be (None, None, 1056) 
threeway_encoder = TimeDistributed(ThreeWay(output_dim=512))
three_way_rslt = threeway_encoder(aggre_threeway_inputs) # should be (None, None, 512) 
logits = Dense(365, activation='softmax')(three_way_rslt) # should be (None, None, 365)
self.model = keras.Model(inputs=multiple_inputs, outputs=logits)

By removing below two lines (by not making it go through the embedding layer) , the code works without any issues. In this case, dimension of aggre_threeway_inputs = Concatenate()([user_input_for_TD, all_areas_lstm]) is (None, None, 1025).

user_input_for_TD = TimeDistributed(user_embedding_layer)(user_input_for_TD)
user_input_for_TD = K.squeeze(user_input_for_TD, axis=-2)

回答1:

I solved it by using the Lambda layer with indexing, instead of K.squeeze function.

from keras.layers import Lambda
>>> K.int_shape(user_input_for_TD)
(None, None, 1, 32)
>>> K.int_shape(Lambda(lambda x: x[:, :, 0, :])(user_input_for_TD))
(None, None, 32)

来源：https://stackoverflow.com/questions/55393031/keras-how-to-remove-useless-dimension-without-hurting-the-computation-graph

标签

keras