Integrating BERT sentence embedding into a siamese LSTM network

问题

I am working on a text similarity project and I wanted to experiment with a siamese LSTM network. I am working on modifying this implementation https://amitojdeep.github.io/amitoj-blogs/2017/12/31/semantic-similarity.html . The code is based on using Word2Vec word embeddings and I wanted to replace that with BERT sentence embeddings https://github.com/imgarylai/bert-embedding

The resulting matrix has column 1 with the input sentence strings, column 2 with each cell containing the corresponding embedding matrix (num_words, 768). My understanding is that using this embedding matrix I can simply skip the embedding layer step and substitute it directly into the next layer. Which is what I tried doing below.

# The visible layer

left_input = Input(shape=(max_seq_length,), dtype='int32')
right_input = Input(shape=(max_seq_length,), dtype='int32')

#embedding_layer = Embedding(len(embeddings), embedding_dim, weights=[embeddings], input_length=max_seq_length, trainable=False)
#embedding layer modified to use embeddings from BERT

# Embedded version of the inputs  
encoded_left_full = bert_embedding(left_input) 
encoded_left = encoded_left_full[0][1]
encoded_right_full = bert_embedding(right_input)
encoded_right = encoded_right_full[0][1]

I am getting the below error message:

TypeError: object of type 'Tensor' has no len().

My understanding is that as I have taken an input tensor but the bert_embedding function uses the string directly, there is a type mismatch. Please advise me on possible solutions like directly taking the string as an input or any other modifications.

来源：https://stackoverflow.com/questions/56980868/integrating-bert-sentence-embedding-into-a-siamese-lstm-network