rnn | 易学教程

How do we use LSTM to classify sequences?

阅读更多关于 How do we use LSTM to classify sequences?

问题 LSTM is good for predicting what is going to happen after a sequence, but I assume that we have many sequences and that each sequence corresponds to a class label. How can we use LSTM to classify these sequences? 回答1: LSTM can be used for prediction as well as classification tasks. For classification, you can follow most commonly used architectures that I have described below. However, you can build your own model depending on your requirement. As the output of LSTM (Here I explain dynamic

PyTorch: DecoderRNN: RuntimeError: input must have 3 dimensions, got 2

阅读更多关于 PyTorch: DecoderRNN: RuntimeError: input must have 3 dimensions, got 2

问题 I am building a DecoderRNN using PyTorch (This is an image-caption decoder): class DecoderRNN(nn.Module): def __init__(self, embed_size, hidden_size, vocab_size): super(DecoderRNN, self).__init__() self.hidden_size = hidden_size self.gru = nn.GRU(embed_size, hidden_size, hidden_size) self.softmax = nn.LogSoftmax(dim=1) def forward(self, features, captions): print (features.shape) print (captions.shape) output, hidden = self.gru(features, captions) output = self.softmax(self.out(output[0]))

Tensorflow: how to obtain intermediate cell states (c) from LSTMCell using dynamic_rnn?

阅读更多关于 Tensorflow: how to obtain intermediate cell states (c) from LSTMCell using dynamic_rnn?

问题 By default, function dynamic_rnn outputs only hidden states (known as m ) for each time point which can be obtained as follows: cell = tf.contrib.rnn.LSTMCell(100) rnn_outputs, _ = tf.nn.dynamic_rnn(cell, inputs=inputs, sequence_length=sequence_lengths, dtype=tf.float32) Is there a way get intermediate (not final) cell states ( c ) in addition? A tensorflow contributor mentions that it can be done with a cell wrapper: class Wrapper(tf.nn.rnn_cell.RNNCell): def __init__(self, inner_cell):

Error in fitting an RNN LSTM model

阅读更多关于 Error in fitting an RNN LSTM model

问题 I am trying to create an RNN LSTM model for a binary classification using the following code alldataset = np.loadtxt("FinalKNEEALL.txt", delimiter=",") num_classes = 2 num_of_sam = alldataset.shape[0] labels = np.ones((num_of_sam,), dtype='int64') labels[0:958943]=0 labels[958943:1917887]=1 Y = np_utils.to_categorical(labels,num_classes) x,y = shuffle (alldataset,Y, random_state=2) x_train,x_test, y_train,y_test = train_test_split(x,y, test_size=0.3, random_state=4) print(x_train.shape) print

Euclidean distance loss function for RNN (keras)

阅读更多关于 Euclidean distance loss function for RNN (keras)

问题 I want to set Euclidean distance as a loss function for LSTM or RNN. What output should such function have: float, (batch_size) or (batch_size, timesteps)? Model input X_train is (n_samples, timesteps, data_dim). Y_train has the same dimensions. Example code: def euc_dist_keras(x, y): return K.sqrt(K.sum(K.square(x - y), axis=-1, keepdims=True)) model = Sequential() model.add(SimpleRNN(n_units, activation='relu', input_shape=(timesteps, data_dim), return_sequences=True)) model.add(Dense(n

Python - RNN LSTM model low accuracy

阅读更多关于 Python - RNN LSTM model low accuracy

问题 I have tried to build LSTM model with this sample of dataset (patient number, time in mill/sec., normalization of X Y and Z, kurtosis, skewness, pitch, roll and yaw, label) respectively. 1,15,-0.248010047716,0.00378335508419,-0.0152548459993,-86.3738760481,0.872322164158,-3.51314800063,0 1,31,-0.248010047716,0.00378335508419,-0.0152548459993,-86.3738760481,0.872322164158,-3.51314800063,0 1,46,-0.267422664673,0.0051143782875,-0.0191247001961,-85.7662354031,1.0928406847,-4.08015176908,0 1,62,-0

CNN和RNN中如何引入BatchNorm

阅读更多关于 CNN和RNN中如何引入BatchNorm

转载： https://blog.csdn.net/malefactor/article/details/51549771 Batch Normalization的基本思路和价值在之前一篇文章“Batch Normalization导读”介绍了，此处不赘述，背景知识请参考上面文章。看到BN后，很明显能够看到这等于往传统的神经网络中增加了一个BN层，而且位置处于神经元非线性变换前，基本大多数网络结构都能很自然地融合进去，于是很自然的想法就是：如果用在CNN或者RNN效果会如何？是否也会增加收敛速度以及模型分类性能？CNN的使用方式在原始的Batch Normalization论文就说了，RNN则有相关后续研究跟进，目前看还没有特别明确的结论。 |CNN的BatchNorm CNN和DNN不一样，某个卷积层包含多个FilterMap，而每个Filter Map其实是参数共享的，侦测同一类特征，是通过在输入图像上的局部扫描的方式遍历覆盖整个输入图像的，但是单个Filter Map本身可能是二维甚至多维的，如果是二维的，那么包含p*q个神经元。那么此时要应用BN其实有两种选择：一种是把一个FilterMap看成一个整体，可以想象成是一个Filter Map对应DNN隐层中的一个神经元，所以一个Filter Map的所有神经元共享一个Scale和Shift参数，Mini

PyTorch: passing numpy array for weight initialization

阅读更多关于 PyTorch: passing numpy array for weight initialization

问题 I'd like to initialize the parameters of RNN with np arrays. In the following example, I want to pass w to the parameters of rnn . I know pytorch provides many initialization methods like Xavier, uniform, etc., but is there way to initialize the parameters by passing numpy arrays? import numpy as np import torch as nn rng = np.random.RandomState(313) w = rng.randn(input_size, hidden_size).astype(np.float32) rnn = nn.RNN(input_size, hidden_size, num_layers) 回答1: First, let's note that nn.RNN

Community structure enhanced cascade prediction 笔记

阅读更多关于 Community structure enhanced cascade prediction 笔记

目录一、摘要二、杂记三、模型思想四、实验五、其他六、参考文献一、摘要深度学习不用去手工提取特征，但是现有深度模型没有在传播预测任务中使用社区结构。所以提出一个CS-RNN框架，把社区在传播中的影响考虑在内，做传播预测。二、杂记贡献： ①引入community structure information；②CS-RNN模型包含社区结构标签预测层(community structure labels prediction layer)，可以预测下一个活跃节点的社区结构标签；③参数健壮、结果好；（PS：原文中3、4点被总结到此处第3点了，单从此处来看创新并不是很吸引人。）参考文献结论： strong community 通过加强局部和社区内部的传播，有利于信息进行全局传播。文章缺点：参考文献列举了随着神经网络发展产生的一些深度模型，但是并没有描述这些深度模型的侧重点或者缺点。（PS：那如何突出自己工作的重要性）三、模型思想 3.1 问题定义： Network： \(G=(V,E)\) ， \(V\) 是顶点代表用户， \(E\) 是边代表用户间关系； Cascade： \(S=\{\left(t_{i}, v_{i}\right) | v_{i} \in V, t_{i} \in[0,+\infty), t_i \le t_{i+1}, i=1,2,...,N

Sequence to Sequence Learning with Neural Networks（翻译）

阅读更多关于 Sequence to Sequence Learning with Neural Networks（翻译）

Sequence to Sequence Learning with Neural Networks 摘要本文中提出了一种通用的端到端的学习方式，对序列结构做出了做小的假设。我们的方法是使用多层LSTM将输入序列映射到固定维度的向量，然后使用另一个深层的LSTM 将这个向量解码到目标序列。LSTM还学习到合理的短语和句子的表达，即：对语序敏感而对主动语态和被动语态并不敏感。最后，我们发现翻转源句子中的单词的顺序可以明显的提高LSTM的性能，因为这样做会在源语句和目标语句之间引入许多短期相关性，从而使优化问题变得更容易。 1 简介 DNN功能强大，并且实现了卓越的性能。虽然DNN灵活且强大，但是只能应用在输入和目标可以用固定维度的向量合理的编码的问题上。这是一个严重的限制，因为许多重要的表达最好是用长度不定的序列表达。序列对DNN构成了一个挑战，因为序列要求输入和输出的维度已知并且固定。本文中, 我们提出LSTM结构直接应用可以解决一般的序列到序列的问题。这个想法是用一个LSTM读取输入序列，一次一个时间步，去获得大的固定向量的表示，然后使用另一个输出序列在从该项量中提取出输出序列。第二个LSTM除了它取决于输入序列，其本质上就是一个RNN语言模型。LSTM可以成功在数据上学习长依赖的能力使其成为了该应用的自然选择，因为输入和相应的输出之间存在着相当的时间延迟。注：图 1

订阅 rnn