lstm

How to correctly give inputs to Embedding, LSTM and Linear layers in PyTorch?

北慕城南 提交于 2019-12-03 02:46:36
问题 I need some clarity on how to correctly prepare inputs for batch-training using different components of the torch.nn module. Specifically, I'm looking to create an encoder-decoder network for a seq2seq model. Suppose I have a module with these three layers, in order: nn.Embedding nn.LSTM nn.Linear nn.Embedding Input: batch_size * seq_length Output: batch_size * seq_length * embedding_dimension I don't have any problems here, I just want to be explicit about the expected shape of the input and

Tensorflow dynamic RNN (LSTM): how to format input?

匿名 (未验证) 提交于 2019-12-03 02:45:02
可以将文章内容翻译成中文,广告屏蔽插件可能会导致该功能失效(如失效,请关闭广告屏蔽插件后再试): 问题: I have been given some data of this format and the following details: person1, day1, feature1, feature2, ..., featureN, label person1, day2, feature1, feature2, ..., featureN, label ... person1, dayN, feature1, feature2, ..., featureN, label person2, day1, feature1, feature2, ..., featureN, label person2, day2, feature1, feature2, ..., featureN, label ... person2, dayN, feature1, feature2, ..., featureN, label ... there is always the same number of features but each feature might be a 0 representing nothing there is a varying amount of days

Tensorflow LSTM Regularization

匿名 (未验证) 提交于 2019-12-03 02:41:02
可以将文章内容翻译成中文,广告屏蔽插件可能会导致该功能失效(如失效,请关闭广告屏蔽插件后再试): 问题: I was wondering how one can implement l1 or l2 regularization within an LSTM in TensorFlow? TF doesn't give you access to the internal weights of the LSTM, so I'm not certain how one can calculate the norms and add it to the loss. My loss function is just RMS for now. The answers here don't seem to suffice. 回答1: The answers in the link you mentioned are the correct way to do it. Iterate through tf.trainable_variables and find the variables associated with your LSTM. An alternative, more complicated and possibly more brittle approach is to re

Using BI LSTM CTC Tensorflow Model in Android

匿名 (未验证) 提交于 2019-12-03 02:38:01
可以将文章内容翻译成中文,广告屏蔽插件可能会导致该功能失效(如失效,请关闭广告屏蔽插件后再试): 问题: TL;DR, I want to know how to use a bi-lstm-ctc tensorflow model in an android application. I have succeeded in training my bi-lstm-ctc tensorflow model and now I want to use it for my handwriting recognition android application. Here's the part of the code that defines the graph I used: self.inputs = tf.placeholder(tf.float32, [None, None, network_config.num_features], name="input") self.labels = tf.sparse_placeholder(tf.int32, name="label") self.seq_len = tf.placeholder(tf.int32, [None], name="seq_len_input") logits = self._bidirectional

Reuse Reusing Variable of LSTM in Tensorflow

匿名 (未验证) 提交于 2019-12-03 02:29:01
可以将文章内容翻译成中文,广告屏蔽插件可能会导致该功能失效(如失效,请关闭广告屏蔽插件后再试): 问题: I'm trying to make RNN using LSTM . I made LSTM model, and after it, there is two DNN network, and one regression output layer. I trained my data, and the final training loss become about 0.009 . However, when i applied the model to test data, the loss become about 0.5 . The 1th epoch training loss is about 0.5 . So, I think the trained variable do not used in test model. The only difference between training and test model is batch size. Trainning Batch = 100~200 , Test Batch Size = 1 . in main function i made LSTM instance. In LSTM

How do you pass video features from a CNN to an LSTM?

匿名 (未验证) 提交于 2019-12-03 02:20:02
可以将文章内容翻译成中文,广告屏蔽插件可能会导致该功能失效(如失效,请关闭广告屏蔽插件后再试): 问题: After you pass a video frame through a convnet and get an output feature map, how do you pass that data into an LSTM? Also, how do you pass multiple frames to the LSTM thru the CNN? In other works I want to process video frames with an CNN to get the spatial features. Then I want pass these features to an LSTM to do temporal processing on the spatial features. How do I connect the LSTM to the video features? For example if the input video is 56x56 and then when passed through all of the CNN layers, say it comes out as 20: 5x5's. How are

LSTM Followed by Mean Pooling

匿名 (未验证) 提交于 2019-12-03 02:20:02
可以将文章内容翻译成中文,广告屏蔽插件可能会导致该功能失效(如失效,请关闭广告屏蔽插件后再试): 问题: I'm using Keras 1.0. My problem is identical to this one ( How to implement a Mean Pooling layer in Keras ), but the answer there does not seem to be sufficient for me. I want to implement this network: The following code does not work: sequence = Input(shape=(max_sent_len,), dtype='int32') embedded = Embedding(vocab_size, word_embedding_size)(sequence) lstm = LSTM(hidden_state_size, activation='sigmoid', inner_activation='hard_sigmoid', return_sequences=True)(embedded) pool = AveragePooling1D()(lstm) output = Dense(1, activation='sigmoid')

LSTM Autoencoder

匿名 (未验证) 提交于 2019-12-03 02:16:02
可以将文章内容翻译成中文,广告屏蔽插件可能会导致该功能失效(如失效,请关闭广告屏蔽插件后再试): 问题: I'm trying to build a LSTM autoencoder with the goal of getting a fixed sized vector from a sequence, which represents the sequence as good as possible. This autoencoder consists of two parts: LSTM Encoder: Takes a sequence and returns an output vector ( return_sequences = False ) LSTM Decoder: Takes an output vector and returns a sequence ( return_sequences = True ) So, in the end, the encoder is a many to one LSTM and the decoder is a one to many LSTM. Image source: Andrej Karpathy On a high level the coding looks like this (similar as

Keras LSTM input dimension setting

匿名 (未验证) 提交于 2019-12-03 02:14:01
可以将文章内容翻译成中文,广告屏蔽插件可能会导致该功能失效(如失效,请关闭广告屏蔽插件后再试): 问题: I was trying to train a LSTM model using keras but I think I got something wrong here. I got an error of ValueError: Error when checking input: expected lstm_17_input to have 3 dimensions, but got array with shape (10000, 0, 20) while my code looks like model = Sequential() model.add(LSTM(256, activation="relu", dropout=0.25, recurrent_dropout=0.25, input_shape=(None, 20, 64))) model.add(Dense(1, activation="sigmoid")) model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy']) model.fit(X_train, y_train, batch_size

Stateful LSTM: When to reset states?

匿名 (未验证) 提交于 2019-12-03 02:01:02
可以将文章内容翻译成中文,广告屏蔽插件可能会导致该功能失效(如失效,请关闭广告屏蔽插件后再试): 问题: Given X with dimensions (m samples, n sequences, and k features) , and y labels with dimensions (m samples, 0/1) : Suppose I want to train a stateful LSTM (going by keras definition, where "stateful = True" means that cell states are not reset between sequences per sample -- please correct me if I'm wrong!), are states supposed to be reset on a per epoch basis or per sample basis? Example: for e in epoch: for m in X.shape[0]: #for each sample for n in X.shape[1]: #for each sequence #train_on_batch for model... #model.reset_states() (1) I