bert-language-model

问题 I earlier posted this question. I wanted to get embedding similar to this youtube video, time 33 minutes onward. 1) I dont think that the embedding that i am getting from CLS token are similar to what is shown in the youtube video. I tried to perform semantic similarity and got horrible results. Could someone confirm whether embedding that i am getting are similar to embedding mentioned at 35.27 mark of the video? 2) If the answer of the above question is 'not similar' then how could i get

问题 I have created a binary classifier with Tensorflow BERT language model. Here is the link. After the model is trained, it saves the model and produces the following files. Prediction code. from tensorflow.contrib import predictor #MODEL_FILE = 'graph.pbtxt' with tf.Session() as sess: predict_fn = predictor.from_saved_model(f'/content/drive/My Drive/binary_class/bert/graph.pbtxt') predictions = predict_fn(pred_sentences) print(predictions) Error OSError: SavedModel file does not exist at:

问题 I have created a binary classifier with Tensorflow BERT language model. Here is the link. After the model is trained, it saves the model and produces the following files. Prediction code. from tensorflow.contrib import predictor #MODEL_FILE = 'graph.pbtxt' with tf.Session() as sess: predict_fn = predictor.from_saved_model(f'/content/drive/My Drive/binary_class/bert/graph.pbtxt') predictions = predict_fn(pred_sentences) print(predictions) Error OSError: SavedModel file does not exist at:

问题 I am doing a long text classification task, which has more than 10000 words in doc, I am planing to use Bert as a paragraph encoder, then feed the embeddings of paragraph to BiLSTM step by step. The network is as below: Input: (batch_size, max_paragraph_len, max_tokens_per_para,embedding_size) bert layer: (max_paragraph_len,paragraph_embedding_size) lstm layer: ??? output layer: (batch_size,classification_size) How to implement it with keras? I am using keras's load_trained_model_from

问题 BERT output is not deterministic. I expect the output values are deterministic when I put a same input, but my bert model the values are changing. Sounds awkwardly, the same value is returned twice, once. That is, once another value comes out, the same value comes out and it repeats. How I can make the output deterministic? let me show snippets of my code. I use the model as below. For the BERT implementation, I use huggingface implemented BERT pytorch implementation. which is quite fameous

问题 I've problems integrating Bert Embedding Layer in a BiLSTM model for word sense disambiguation task, Windows 10 Python 3.6.4 TenorFlow 1.12 Keras 2.2.4 No virtual environments were used PyCharm Professional 2019.2 The whole script import os import yaml import numpy as np from argparse import ArgumentParser import tensorflow as tf import tensorflow_hub as hub from tensorflow.keras.layers import (LSTM, Add, Bidirectional, Dense, Input, TimeDistributed, Embedding) from tensorflow.keras