lstm

I'm going through this tutorial on RNNs/LSTMs and I'm having quite a hard time understanding stateful LSTMs. My questions are as follows : 1. Training batching size In the Keras docs on RNNs , I found out that the hidden state of the sample in i -th position within the batch will be fed as input hidden state for the sample in i -th position in the next batch. Does that mean that if we want to pass the hidden state from sample to sample we have to use batches of size 1 and therefore perform online gradient descent? Is there a way to pass the hidden state within a batch of size >1 and perform

I'm trying to do binary LSTM classification using theano. I have gone through the example code however I want to build my own. I have a small set of "Hello" & "Goodbye" recordings that I am using. I preprocess these by extracting the MFCC features for them and saving these features in a text file. I have 20 speech files(10 each) and I am generating a text file for each word, so 20 text files that contains the MFCC features. Each file is a 13x56 matrix. My problem now is: How do I use this text file to train the LSTM? I am relatively new to this. I have gone through some literature on it as

My input is simply a csv file with 339732 rows and two columns : the first being 29 feature values, i.e. X the second being a binary label value, i.e. Y I am trying to train my data on a stacked LSTM model: data_dim = 29 timesteps = 8 num_classes = 2 model = Sequential() model.add(LSTM(30, return_sequences=True, input_shape=(timesteps, data_dim))) # returns a sequence of vectors of dimension 30 model.add(LSTM(30, return_sequences=True)) # returns a sequence of vectors of dimension 30 model.add(LSTM(30)) # return a single vector of dimension 30 model.add(Dense(1, activation='softmax')) model

LSTM记录贴 得平常再做一些任务都会用到LSTM（长短时记忆网络）、GRU以及一些变形的RNN模型，虽然平时用的多，但近来看面经，发现有些问题都忘了，回过头看了又忘，所以准备好好的手推一遍并且记录下来； 先上图 其实如果推理过LSTM的话，基本上这图就没啥用，因为这图说实话画的略显复杂。在真的面试遇到手推LSTM，方法应该是从三个门出发点去反向画图。 先说主人公三个门，分别是遗忘门（f）、输入门（i）、输出门（o） 还有出场的人员包括上个时间步细胞状态（ct-1）、本时刻的输入（xt）、上一时间步的输入（ht-1）、临时细胞状态（c~） 以及lstm每个unit特有的输入框架，相比普通RNN多了一个c而已 那么可以开始推导了 首先要确定，每个lstm会有三个输入（ct-1、ht-1、xt），两个输出（ct、ht） 而ht-1与xt都是向量，在论文中xt与ht-1是直接横向拼接在一起的（可以有不同的结合方式，但是基本是绑定的），因为两个向量都是定长，所以在经过每个门的线性变换矩阵输出也是定长。 那么开始第一个门，决定我们会从细胞状态中丢弃什么信息，所以叫遗忘门，f= sigmoid（[xt,ht-1]Wf+bf），输出的是一个概率，遗忘概率。那么遗忘的对象是谁？那必然是上个时刻输入的细胞状态ct-1 那么可以画出图 输入门（或者更新门）i = sigmoid（[xt,ht-1]Wi

LSTM/RNN can be used for text generation. This shows way to use pre-trained GloVe word embeddings for Keras model. How to use pre-trained Word2Vec word embeddings with Keras LSTM model? This post did help. How to predict / generate next word when the model is provided with the sequence of words as its input? Sample approach tried: # Sample code to prepare word2vec word embeddings import gensim documents = ["Human machine interface for lab abc computer applications", "A survey of user opinion of computer system response time", "The EPS user interface management system", "System and human system

I was trying to train a LSTM model using keras but I think I got something wrong here. I got an error of ValueError: Error when checking input: expected lstm_17_input to have 3 dimensions, but got array with shape (10000, 0, 20) while my code looks like model = Sequential() model.add(LSTM(256, activation="relu", dropout=0.25, recurrent_dropout=0.25, input_shape=(None, 20, 64))) model.add(Dense(1, activation="sigmoid")) model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy']) model.fit(X_train, y_train, batch_size=batch_size, epochs=10) where X_train has a shape of