lstm | 易学教程

开始使用Keras函数API(翻译整理自Keras英文文档)

阅读更多关于开始使用Keras函数API(翻译整理自Keras英文文档)

Keras函数API是定义复杂模型(如多输出模型、有向无环图或具有共享层的模型)的方法。第一个例子:密集连接的网络对于实现这样的网络，顺序模型可能是更好的选择，但它有助于从一些非常简单的东西开始。 from keras.layers import Input, Dense from keras.models import Model # This returns a tensor inputs = Input(shape=(784,)) #层实例可以在一个tensor上调用，并返回一个tensor output_1 = Dense(64, activation='relu')(inputs) output_2 = Dense(64, activation='relu')(output_1) predictions = Dense(10, activation='softmax')(output_2) # 构建一个模型包括 # 输入层和三个dense model = Model(inputs=inputs, outputs=predictions) model.compile(optimizer='rmsprop', loss='categorical_crossentropy', metrics=['accuracy']) model.fit(data, labels) #

LSTM——长短时记忆网络

阅读更多关于 LSTM——长短时记忆网络

　　LSTM(Long Short-term Memory)，长短时记忆网络是1997年Hochreiter和Schmidhuber为了解决预测位置与相关信息之间的间隔增大或者复杂语言场景中，有用信息间隔有大有小、长短不一，造成循环神经网络性能受到限制而提出的。　　LSTM是RNN的一种特殊类型，它可以学习长期依赖的信息。与单一RNN不同，LSTM网络结构是一种拥有3个”门”结构的特殊网络结构，这个特殊设计可以避免长期依赖问题。　　下面介绍LSTM网络结构：　　　　原始的RNN隐藏层只有一个状态h，它对于短期的输入非常敏感。LSTM网络增加了一个状态c，让它来保存长期的状态。新增的状态c，称为单元状态。将（b)按照时间维度展开，如下图所示：　　如上图可以看出，在t时刻，LSTM网络的输入有3个，即当前时刻网络状态的输入值x t 、上一时刻LSTM网络的输入值h t-1 以及上一时刻的单元状态c t-1 ；LSTM网络的输出有两个，即当前时刻LSTM网络输出值h t 和当前时刻的单元状态c t 。注意，x、c、h都是向量。　　LSTM网络的关键，就是怎样控制长期状态c。在这里，LSTM的思路是使用三个控制开关：第一个开关，负责控制保存长期状态c；第二个开关，负责控制把即时状态输入到长期状态c；第三个开关，负责控制是否把长期状态c作为当前的LSTM网络的输出。　

Reuse Reusing Variable of LSTM in Tensorflow

阅读更多关于 Reuse Reusing Variable of LSTM in Tensorflow

问题 I'm trying to make RNN using LSTM . I made LSTM model, and after it, there is two DNN network, and one regression output layer. I trained my data, and the final training loss become about 0.009 . However, when i applied the model to test data, the loss become about 0.5 . The 1th epoch training loss is about 0.5 . So, I think the trained variable do not used in test model. The only difference between training and test model is batch size. Trainning Batch = 100~200 , Test Batch Size = 1 . in

【tensorflow】static_rnn与dynamic_rnn的区别

阅读更多关于【tensorflow】static_rnn与dynamic_rnn的区别

static_rnn和dynamic_rnn的区别主要在于实现不同。 static_rnn会把RNN展平，用空间换时间。 gpu会吃不消（个人测试结果） dynamic_rnn则是使用for或者while循环。调用static_rnn实际上是生成了rnn按时间序列展开之后的图。打开tensorboard你会看到sequence_length个rnn_cell stack在一起，只不过这些cell是share weight的。因此，sequence_length就和图的拓扑结构绑定在了一起，因此也就限制了每个batch的sequence_length必须是一致。调用dynamic_rnn不会将rnn展开，而是利用tf.while_loop这个api，通过Enter, Switch, Merge, LoopCondition, NextIteration等这些control flow的节点，生成一个可以执行循环的图（这个图应该还是静态图，因为图的拓扑结构在执行时是不会变化的）。在tensorboard上，你只会看到一个rnn_cell, 外面被一群control flow节点包围着。对于dynamic_rnn来说，sequence_length仅仅代表着循环的次数，而和图本身的拓扑没有关系，所以每个batch可以有不同sequence_length。 static_rnn 导包

Stock price predictions of keras multilayer LSTM model converge to a constant value

阅读更多关于 Stock price predictions of keras multilayer LSTM model converge to a constant value

问题 I've made a multilayer LSTM model that uses regression to predict next frame's values of the data. The model finishes after 20 epochs. I then get some predictions and compare them to my ground truth values. As you can see them in the picture above, predictions converge to a constant value. I don't know why this happens. Here is my model so far: from keras.models import Sequential from keras.layers.core import Dense, Activation, Dropout from keras.layers import LSTM, BatchNormalization from

Custom Data Generator for Keras LSTM with TimeSeriesGenerator

阅读更多关于 Custom Data Generator for Keras LSTM with TimeSeriesGenerator

问题 So I'm trying to use Keras' fit_generator with a custom data generator to feed into an LSTM network. What works To illustrate the problem, I have created a toy example trying to predict the next number in a simple ascending sequence, and I use the Keras TimeseriesGenerator to create a Sequence instance: WINDOW_LENGTH = 4 data = np.arange(0,100).reshape(-1,1) data_gen = TimeseriesGenerator(data, data, length=WINDOW_LENGTH, sampling_rate=1, batch_size=1) I use a simple LSTM network: data_dim =

RNN（LSTM)的Tensorflow实现源码

阅读更多关于 RNN（LSTM)的Tensorflow实现源码

# Copyright 2015 The TensorFlow Authors. All Rights Reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the

Python keras how to change the size of input after convolution layer into lstm layer

阅读更多关于 Python keras how to change the size of input after convolution layer into lstm layer

问题 I have a problem with the connection between convolution layer and lstm layer. The data is of shape(75,5) where there is 75 timesteps x 5 data points for each time step. What I want to do is do a convolution on (75x5), get new convolved (75x5) data and feed that data into lstm layer. However, it does not work because the shape of output of convolution layer has number of filters which I do not need. And therefore the shape of convolution layer output is (1,75,5) and input needed for lstm

How do I create padded batches in Tensorflow for tf.train.SequenceExample data using the DataSet API?

阅读更多关于 How do I create padded batches in Tensorflow for tf.train.SequenceExample data using the DataSet API?

问题 For training an LSTM model in Tensorflow , I have structured my data into a tf.train.SequenceExample format and stored it into a TFRecord file . I would now like to use the new DataSet API to generate padded batches for training . In the documentation there is an example for using padded_batch, but for my data I can't figure out what the value of padded_shapes should be. For reading the TFrecord file into the batches I have written the following Python code: import math import tensorflow as

LSTM with Keras for mini-batch training and online testing

阅读更多关于 LSTM with Keras for mini-batch training and online testing

问题 I would like to implement an LSTM in Keras for streaming time-series prediction -- i.e., running online, getting one data point at a time. This is explained well here, but as one would assume, the training time for an online LSTM can be prohibitively slow. I would like to train my network on mini-batches, and test (run prediction) online. What is the best way to do this in Keras? For example, a mini-batch could be a sequence of 1000 data values ( [33, 34, 42, 33, 32, 33, 36, ... 24, 23] )

订阅 lstm