lstm | 易学教程

Tensor Flow - LSTM - 'Tensor' object not iterable

阅读更多关于 Tensor Flow - LSTM - 'Tensor' object not iterable

问题 Hi I am using the following function for lstm rnn cell. def LSTM_RNN(_X, _istate, _weights, _biases): # Function returns a tensorflow LSTM (RNN) artificial neural network from given parameters. # Note, some code of this notebook is inspired from an slightly different # RNN architecture used on another dataset: # https://tensorhub.com/aymericdamien/tensorflow-rnn # (NOTE: This step could be greatly optimised by shaping the dataset once # input shape: (batch_size, n_steps, n_input) _X = tf

Running LSTM with multiple GPUs gets “Input and hidden tensors are not at the same device”

阅读更多关于 Running LSTM with multiple GPUs gets “Input and hidden tensors are not at the same device”

问题 I am trying to train a LSTM layer in pytorch. I am using 4 GPUs. When initializing, I added the .cuda() function move the hidden layer to GPU. But when I run the code with multiple GPUs I am getting this runtime error : RuntimeError: Input and hidden tensors are not at the same device I have tried to solve the problem by using .cuda() function in the forward function like below : self.hidden = (self.hidden[0].type(torch.FloatTensor).cuda(), self.hidden[1].type(torch.FloatTensor).cuda()) This

Integrating BERT sentence embedding into a siamese LSTM network

阅读更多关于 Integrating BERT sentence embedding into a siamese LSTM network

问题 I am working on a text similarity project and I wanted to experiment with a siamese LSTM network. I am working on modifying this implementation https://amitojdeep.github.io/amitoj-blogs/2017/12/31/semantic-similarity.html . The code is based on using Word2Vec word embeddings and I wanted to replace that with BERT sentence embeddings https://github.com/imgarylai/bert-embedding The resulting matrix has column 1 with the input sentence strings, column 2 with each cell containing the

02 序列模型问题

阅读更多关于 02 序列模型问题

序列模型问题给定一个序列, 预测下一个出现的item. 如字迹预测, 语句单词预测, 行为预测等等. LSTM 网络 Long Short Term 网络，一般就叫做 LSTM ，是一种 RNN 特殊的类型，可以学习长期依赖信息。LSTM 通过刻意的设计来避免长期依赖问题。记住长期的信息在实践中是 LSTM 的默认行为，而非需要付出很大代价才能获得的能力！ LSTM前向传播算法来源： https://www.cnblogs.com/lee3258/p/11993972.html

02 序列模型问题

阅读更多关于 02 序列模型问题

How to setup input shape for 1dCNN+LSTM network (Keras)?

阅读更多关于 How to setup input shape for 1dCNN+LSTM network (Keras)?

I have the following idea to implement: Input -> CNN-> LSTM -> Dense -> Output The Input has 100 time steps, each step has a 64-dimensional feature vector A Conv1D layer will extract features at each time step. The CNN layer contains 64 filters, each has length 16 taps. Then, a maxpooling layer will extract the single maximum value of each convolutional output, so a total of 64 features will be extracted at each time step. Then, the output of the CNN layer will be fed into an LSTM layer with 64 neurons. Number of recurrence is the same as time step of input, which is 100 time steps. The LSTM

seq2seq prediction for time series

阅读更多关于 seq2seq prediction for time series

问题 I want to make a Seq2Seq model for reconstruction purpose. I want a model trained to reconstruct the normal time-series and it is assumed that such a model would do badly to reconstruct the anomalous time-series having not seen them during training. I have some gaps in my code and also in the understanding. I took this as an orientation and did so far: traindata: input_data.shape(1000,60,1) and target_data.shape(1000,50,1) with target data being the same training data only in reversed order

Keras LSTM dense layer multidimensional input

阅读更多关于 Keras LSTM dense layer multidimensional input

I'm trying to create a keras LSTM to predict time series. My x_train is shaped like 3000,15,10 (Examples, Timesteps, Features), y_train like 3000,15,1 and I'm trying to build a many to many model (10 input features per sequence make 1 output / sequence). The code I'm using is this: model = Sequential() model.add(LSTM( 10, input_shape=(15, 10), return_sequences=True)) model.add(Dropout(0.2)) model.add(LSTM( 100, return_sequences=True)) model.add(Dropout(0.2)) model.add(Dense(1, activation='linear')) model.compile(loss="mse", optimizer="rmsprop") model.fit( X_train, y_train, batch_size=512, nb

Adding static data( not changing over time) to sequence data in LSTM

阅读更多关于 Adding static data( not changing over time) to sequence data in LSTM

I am trying to build a model like the following figure. Please see the following image: I want to pass sequence data in LSTM layer and static data (blood group, gender) in another feed forward neural network layer. Later I want to merge them. However, I am confused about the dimenstion here. If my understaning is right(which i depict in the image), how the 5-dimensional sequence data can be merged with 4 dimenstional static data? Also, what is the difference of attention mechanism with this structure? (I found in the KERAS documentation that attention mechanism is an way to add static data

Saving and restoring Keras BLSTM CTC model

阅读更多关于 Saving and restoring Keras BLSTM CTC model

问题 I have been working on speech emotion recognition deep neural network. I have used keras Bidirectional LSTM with CTC loss. i trained the model and saved it model_json = model.to_json() with open("ctc_model.json", "w") as json_file: json_file.write(model_json) model.save_weights("ctc_weights.h5") The problem is i can not use this model to test on on unseen data because the model accepts 4 argument as input and calculates the ctc loss..just build the model and train. so how can i save a model