lstm

I have a keras code written in Sequential style. But I am trying to switch Functional mode because I want to use merge function. But I faced an error below when declaring Model(x, out) . What is wrong in my Functional API code? # Sequential, this is working # out_size==16, seq_len==1 model = Sequential() model.add(LSTM(128, input_shape=(seq_len, input_dim), activation='tanh', return_sequences=True)) model.add(TimeDistributed(Dense(out_size, activation='softmax'))) # Functional API x = Input((seq_len, input_dim)) lstm = LSTM(128, return_sequences=True, activation='tanh')(x) td = TimeDistributed

Trying to build a single output regression model, but there seems to be problem in the last layer inputs = Input(shape=(48, 1)) lstm = CuDNNLSTM(256,return_sequences=True)(inputs) lstm = Dropout(dropouts[0])(lstm) #aux_input auxiliary_inputs = Input(shape=(48, 7)) auxiliary_outputs = TimeDistributed(Dense(4))(auxiliary_inputs) auxiliary_outputs = TimeDistributed(Dense(7))(auxiliary_outputs) #concatenate output = keras.layers.concatenate([lstm, auxiliary_outputs]) output = TimeDistributed(Dense(64, activation='linear'))(output) output = TimeDistributed(Dense(64, activation='linear'))(output)

I have a seq2seq model which is working fine. I want to add an embedding layer in this network which I faced with an error. this is my architecture using pretrained word embedding which is working fine(Actually the code is almost the same code available here , but I want to include the Embedding layer in the model rather than using the pretrained embedding vectors): LATENT_SIZE = 20 inputs = Input(shape=(SEQUENCE_LEN, EMBED_SIZE), name="input") encoded = Bidirectional(LSTM(LATENT_SIZE), merge_mode="sum", name="encoder_lstm")(inputs) encoded = Lambda(rev_ent)(encoded) decoded = RepeatVector

LSTM和双向LSTM讲解及实践 目录 RNN的长期依赖问题 LSTM原理讲解 双向LSTM原理讲解 Keras实现LSTM和双向LSTM 一、RNN的长期依赖问题 在上篇文章中介绍的循环神经网络RNN在训练的过程中会有长期依赖的问题，这是由于RNN模型在训练时会遇到梯度消失(大部分情况)或者梯度爆炸(很少，但对优化过程影响很大)的问题。对于梯度爆炸是很好解决的，可以使用梯度修剪(Gradient Clipping)，即当梯度向量大于某个阈值，缩放梯度向量。但对于梯度消失是很难解决的。所谓的梯度消失或梯度爆炸是指训练时计算和反向传播，梯度倾向于在每一时刻递减或递增，经过一段时间后，梯度就会收敛到零(消失)或发散到无穷大(爆炸)。简单来说，长期依赖的问题就是在每一个时间的间隔不断增大时，RNN会丧失到连接到远处信息的能力。 如下图，随着时间点t的不断递增，当t时刻和0时刻的时间间隔较大的时候，t时刻的记忆ht可能已经丧失了学习连接到远处0时刻的信息的能力了。 假设X0的输入为”我住在深圳”，后面插入了很多其他的句子，然后在Xt输入了“我在市政府上班”。由于X0与Xt相差很远，当RNN输入到Xt时，t时刻的记忆ht已经丧失了X0时保存的信息了。因此在Xt时刻神经网络无法理解到我是在哪一个城市的市政府上班了。 二、LSTM原理讲解 在理论上，RNN绝对可以处理这样的长期依赖问题

I have a sequence input in this shape: (6000, 64, 100, 50) The 6000 is just the number of sample sequences. Each sequences is 64 in length. I plan to fit this input into an LSTM using Keras. I setup my input this way: input = Input(shape=(64, 100, 50)) This gives me an input shape of (?, 64, 100, 50) However, when I put input into my LSTM like so: x = LSTM(256, return_sequences=True)(input) I get this error: Input 0 is incompatible with layer lstm_37: expected ndim=3, found ndim=4 This would have worked if my input shape was something like (?, 64, 100) , but not when I've a 4th dimension. Does

I am trying to build an RNN/LSTM model for binary classification 0 or 1 a sample of my dataset (patient number, time in mill/sec., normalization of X Y and Z, kurtosis, skewness, pitch, roll and yaw, label) respectively. 1,15,-0.248010047716,0.00378335508419,-0.0152548459993,-86.3738760481,0.872322164158,-3.51314800063,0 1,31,-0.248010047716,0.00378335508419,-0.0152548459993,-86.3738760481,0.872322164158,-3.51314800063,0 1,46,-0.267422664673,0.0051143782875,-0.0191247001961,-85.7662354031,1.0928406847,-4.08015176908,0 1,62,-0.267422664673,0.0051143782875,-0.0191247001961,-85.7662354031,1

I have a CNN-RNN model architecture with Bidirectional LSTMS for time series regression problem. My loss does not converge over 50 epochs. Each epoch has 20k samples. The loss keeps bouncing between 0.001 - 0.01 . batch_size=1 epochs = 50 model.compile(loss='mean_squared_error', optimizer='adam') trainingHistory=model.fit(trainX,trainY,epochs=epochs,batch_size=batch_size,shuffle=False) I tried to train the model with incorrectly paired X and Y data for which the loss stays around 0.5 , is it reasonable conclusion that my X and Y have a non linear relationship which can be learned by my model

I'm trying to make RNN using LSTM . I made LSTM model, and after it, there is two DNN network, and one regression output layer. I trained my data, and the final training loss become about 0.009 . However, when i applied the model to test data, the loss become about 0.5 . The 1th epoch training loss is about 0.5 . So, I think the trained variable do not used in test model. The only difference between training and test model is batch size. Trainning Batch = 100~200 , Test Batch Size = 1 . in main function i made LSTM instance. In LSTM innitializer, the model is made. def __init__(self,config