recurrent-neural-network

I'm having trouble with preparing input data for RNN on Keras. Currently, my training data dimension is: (6752, 600, 13) 6752: number of training data 600: number of time steps 13: size of feature vectors (the vector is in float) X_train and Y_train are both in this dimension. I want to prepare this data to be fed into SimpleRNN on Keras. Suppose that we're going through time steps, from step #0 to step #599. Let's say I want to use input_length = 5 , which means that I want to use recent 5 inputs. (e.g. step #10, #11,#12,#13,#14 @ step #14). How should I reshape X_train ? should it be (6752,

There are several classes in tf.nn that relate to RNNs. In the examples I find on the web, tf.nn.dynamic_rnn and tf.nn.rnn seem to be used interchangeably or at least I cannot seem to figure out why one is used in place of the other. What is the difference? Abhishek Mishra From RNNs in Tensorflow, a Practical Guide and Undocumented Features by Denny Britz, published in August 21, 2016. tf.nn.rnn creates an unrolled graph for a fixed RNN length. That means, if you call tf.nn.rnn with inputs having 200 time steps you are creating a static graph with 200 RNN steps. First, graph creation is slow.

In LSTM Network ( Understanding LSTMs ), Why input gate and output gate use tanh? what is the intuition behind this? it is just a nonlinear transformation? if it is, can I change both to another activation function (e.g. ReLU)? Sigmoid specifically, is used as the gating function for the 3 gates(in, out, forget) in LSTM , since it outputs a value between 0 and 1, it can either let no flow or complete flow of information throughout the gates. On the other hand, to overcome the vanishing gradient problem, we need a function whose second derivative can sustain for a long range before going to

I have a dialog corpus like below. And I want to implement a LSTM model which predicts a system action. The system action is described as a bit vector. And a user input is calculated as a word-embedding which is also a bit vector. t1: user: "Do you know an apple?", system: "no"(action=2) t2: user: "xxxxxx", system: "yyyy" (action=0) t3: user: "aaaaaa", system: "bbbb" (action=5) So what I want to realize is "many to many (2)" model. When my model receives a user input, it must output a system action. But I cannot understand return_sequences option and TimeDistributed layer after LSTM. To

Both TensorFlow and Theano do not seem to support cyclic computational graphs, cyclic elements are implemented as recurrent cells with buffer and unrolling (RNN / LSTM cells), but this limitation is mostly related with the computation of back-propagation. I don't have a particular need for computing back-propagation but just the forward propagations. Is there a way to ignore this limitation, or perhaps just to break down arbitrary computational graphs in acyclic components? TensorFlow does support cyclic computation graphs. The tf.while_loop() function allows you to specify a while loop with

EDITED for conciseness. I am trying to build an LSTM model, working off the documentation example at https://keras.io/layers/recurrent/ from keras.models import Sequential from keras.layers import LSTM The following three lines of code (plus comment) are taken directly from the documentation link above: model = Sequential() model.add(LSTM(32, input_dim=64, input_length=10)) # for subsequent layers, not need to specify the input size: model.add(LSTM(16)) ValueError: Input 0 is incompatible with layer lstm_2: expected ndim=3, found ndim=2 I get that error above after executing the second model

I'm doing text tagger using Bidirectional dynamic RNN in tensorflow. After maching input's dimension, I tried to run a Session. this is blstm setting parts: fw_lstm_cell = BasicLSTMCell(LSTM_DIMS) bw_lstm_cell = BasicLSTMCell(LSTM_DIMS) (fw_outputs, bw_outputs), _ = bidirectional_dynamic_rnn(fw_lstm_cell, bw_lstm_cell, x_place, sequence_length=SEQLEN, dtype='float32') and this is runing part: with tf.Graph().as_default(): # Placehoder Settings x_place, y_place = set_placeholder(BATCH_SIZE, EM_DIMS, MAXLEN) # BLSTM Model Building hlogits = tf_kcpt.build_blstm(x_place) # Compute loss loss = tf

So i was stuck with a question on how to combine a CNN with a RNN in Keras. While posting the question someone pointed me out that this is the correct way to approach the problem. Apparently i just overlooked something in the original code, which made me answer my own question. The original problem is as follows: How do you create a model in Keras that has sequences of images as the input, with a CNN 'looking' at each individual image and the sequence of the CNN output being fed into a RNN? To make it more clear: Model one: a CNN that looks at single images. Model two: a RNN that at the

I used to create the RNN network, in version 0.8 of TensorFlow, using: from tensorflow.python.ops import rnn # Define a lstm cell with tensorflow lstm_cell = rnn_cell.BasicLSTMCell(n_hidden, forget_bias=1.0) # Get lstm cell output outputs, states = rnn.rnn(cell=lstm_cell, inputs=x, dtype=tf.float32) rnn.rnn() is not available anymore, and it sounds it has been moved to tf.contrib . What is the exact code to create RNN network out of a BasicLSTMCell ? Or, in the case that I have an stacked LSTM, lstm_cell = tf.contrib.rnn.BasicLSTMCell(hidden_size, forget_bias=0.0) stacked_lstm = tf.contrib.rnn