lstm | 易学教程

LSTM inputs for Tensorflow

阅读更多关于 LSTM inputs for Tensorflow

问题 I'm trying to create an LSTM network in Tensorflow and I'm lost in terminology/basics. I have n time series examples so X = x n , where x i =[[x 1 1 x 1 2 ,x 1 3 ],...,[x m 1 x m 2 ,x m 3 ]] and where x i i is a float. First of all I want to train a model that given the start of a sequence ([x 1 1 x 1 2 ,x 1 3 ]) I can predict the rest of the sequence. Then later I hope to include a classifier to predict which binary class each x i belongs to. So my problem is what do I feed in to the start

How to Merge Numerical and Embedding Sequential Models to treat categories in RNN

阅读更多关于 How to Merge Numerical and Embedding Sequential Models to treat categories in RNN

问题 I would like to build a one layer LSTM model with embeddings for my categorical features. I currently have numerical features and a few categorical features, such as Location, which can't be one-hot encoded e.g. using pd.get_dummies() due to computational complexity, which is what I originally intended to do. Let's visualise an example: Sample Data data = { 'user_id': [1,1,1,1,2,2,3], 'time_on_page': [10,20,30,20,15,10,40], 'location': ['London','New York', 'London', 'New York', 'Hong Kong',

When to stop training neural networks?

阅读更多关于 When to stop training neural networks?

I'm trying to carry out a domain-specific classification research using RNN and have accumulated tens of millions of texts. Since it takes days and even months to run the whole dataset over, I only picked a small portion of it for testing, say 1M texts (80% for training, 20% for validation). I pre-trained the whole corpus with word vectorization and I also applied Dropout to the model to avoid over-fitting. When it trained 60000 text within 12 hrs, the loss had already dropped to a fairly low level with the accuracy 97%. Should I continue or not? Does it help continue with the training? It is

How to handle extremely long LSTM sequence length?

阅读更多关于 How to handle extremely long LSTM sequence length?

问题 I have some data that is sampled at at a very high rate (on the order of hundreds of times per second). This results in a sequence length that is huge (~90,000 samples) on average for any given instance. This entire sequence has a single label. I am trying to use an LSTM neural network to classify new sequences as one of these labels (multiclass classification). However, using an LSTM with a such a large sequence length results in a network that is quite large. What are some methods to

Cannot stack LSTM with MultiRNNCell and dynamic_rnn

阅读更多关于 Cannot stack LSTM with MultiRNNCell and dynamic_rnn

问题 I am trying to build a multivariate time series prediction model. I followed the following tutorial for temperature prediction. http://nbviewer.jupyter.org/github/addfor/tutorials/blob/master/machine_learning/ml16v04_forecasting_with_LSTM.ipynb I want to extend his model to multilayer LSTM model by using following code: cell = tf.contrib.rnn.LSTMCell(hidden, state_is_tuple=True) cell = tf.contrib.rnn.MultiRNNCell([cell] * num_layers,state_is_tuple=True) output, _ = tf.nn.dynamic_rnn(cell=cell

tensorflow static_rnn error: input must be a sequence

阅读更多关于 tensorflow static_rnn error: input must be a sequence

I'm trying to feed my own 3D data to a LSTM. The data have: height = 365, width = 310, time = unknown / inconsistent, consist of 0 and 1, each block of data that produce an output are separated to a single file. import tensorflow as tf import os from tensorflow.contrib import rnn filename = "C:/Kuliah/EmotionRecognition/Train1/D2N2Sur.txt" hm_epochs = 10 n_classes = 12 n_chunk = 443 n_hidden = 500 data = tf.placeholder(tf.bool, name='data') cat = tf.placeholder("float", [None, n_classes]) weights = { 'out': tf.Variable(tf.random_normal([n_hidden, n_classes])) } biases = { 'out': tf.Variable(tf

How to extract the cell state and hidden state from an RNN model in tensorflow?

阅读更多关于 How to extract the cell state and hidden state from an RNN model in tensorflow?

问题 I am new to TensorFlow and have difficulties understanding the RNN module. I am trying to extract hidden/cell states from an LSTM. For my code, I am using the implementation from https://github.com/aymericdamien/TensorFlow-Examples. # tf Graph input x = tf.placeholder("float", [None, n_steps, n_input]) y = tf.placeholder("float", [None, n_classes]) # Define weights weights = {'out': tf.Variable(tf.random_normal([n_hidden, n_classes]))} biases = {'out': tf.Variable(tf.random_normal([n_classes]

Recurrent convolutional BLSTM neural network - arbitrary sequence lengths

阅读更多关于 Recurrent convolutional BLSTM neural network - arbitrary sequence lengths

Using Keras + Theano I successfully made a recurrent bidirectional-LSTM neural network that is capable of training on and classifying DNA sequences of arbitrary lengths, using the following model (for fully working code see: http://pastebin.com/jBLv8B72 ): sequence = Input(shape=(None, ONE_HOT_DIMENSION), dtype='float32') dropout = Dropout(0.2)(sequence) # bidirectional LSTM forward_lstm = LSTM( output_dim=50, init='uniform', inner_init='uniform', forget_bias_init='one', return_sequences=True, activation='tanh', inner_activation='sigmoid', )(dropout) backward_lstm = LSTM( output_dim=50, init=

Predict using data with less time steps (different dimension) using Keras RNN model

阅读更多关于 Predict using data with less time steps (different dimension) using Keras RNN model

According to the nature of RNN, we can get an output of predicted probabilities at every time stamp (unfold in time). Suppose I train an RNN with 5 time steps, each having 6 features. Thus I have to specify the first layer like this(suppose we use a LSTM layer with 20 nodes as the first layer): model.add(LSTM(20, return_sequences=True, input_shape=(5, 6))) And the model works well if I input the same dimension data. However, now I want to use first 3 time steps of the data to get the prediction (input shape will be 3, 6), the same syntax will not be accepted. My question is, is it possible to

Why does RNN always output 1

阅读更多关于 Why does RNN always output 1

I am using Recurrent Neural Networks (RNN) for forecasting, but for some weird reason, it always outputs 1. Here I explain this with a toy example as: Example Consider a matrix M of dimensions (360, 5), and a vector Y which contains rowsum of M . Now, using RNN, I want to predict Y from M . Using rnn R package, I trained model as library(rnn) M <- matrix(c(1:1800),ncol=5,byrow = TRUE) # Matrix (say features) Y <- apply(M,1,sum) # Output equls to row sum of M mt <- array(c(M),dim=c(NROW(M),1,NCOL(M))) # matrix formatting as [samples, timesteps, features] yt <- array(c(Y),dim=c(NROW(M),1,NCOL(Y)