Simplest Lstm training with Keras io

Question

I would like to create the simplest LSTM there is using keras python library.

I have the following code:

import pandas as pd
import numpy as np
from keras.models import Sequential
from keras.layers.core import Dense, Activation
from keras.layers.recurrent import LSTM

X_train = pd.DataFrame( np.array([ [1, 2], [3, 4], [5, 6], [7, 8], [5.1, 6.1], [7.1, 8.1] ]))
y_train = pd.DataFrame( np.array([1, 2, 3, 4, 3, 4]) )
X_test = pd.DataFrame( np.array([ [1.1, 2.1], [3.1, 4.1] ]) )
y_test = pd.DataFrame( np.array([1, 2]) )

model = Sequential()
model.add(LSTM( output_dim = 10, return_sequences=False, input_dim=X_train.shape[1]))
model.add(Dense(input_dim = 10, output_dim=2))
model.add(Activation("linear"))
model.compile(loss="mean_squared_error", optimizer="rmsprop")
model.fit(X_train, y_train)

but does not seem to work...

Epoch 1/100
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/keras/models.py", line 489, in fit
    shuffle=shuffle, metrics=metrics)
  File "/usr/lib/python2.7/site-packages/keras/models.py", line 201, in _fit
    ins_batch = slice_X(ins, batch_ids)
  File "/usr/lib/python2.7/site-packages/keras/models.py", line 55, in slice_X
    return [x[start] for x in X]
  File "/usr/lib/python2.7/site-packages/pandas-0.17.0-py2.7-linux-x86_64.egg/pandas/core/frame.py", line 1908, in __getitem__
    return self._getitem_array(key)
  File "/usr/lib/python2.7/site-packages/pandas-0.17.0-py2.7-linux-x86_64.egg/pandas/core/frame.py", line 1952, in _getitem_array
    indexer = self.ix._convert_to_indexer(key, axis=1)
  File "/usr/lib/python2.7/site-packages/pandas-0.17.0-py2.7-linux-x86_64.egg/pandas/core/indexing.py", line 1121, in _convert_to_indexer
    raise KeyError('%s not in index' % objarr[mask])
KeyError: '[3 4 2 5] not in index'

Can anyone explain me what exactly goes wrong ?

I also tried to Transpose the matrices but that's not it.

Answer 1

First problem I see is using Pandas Dataframe. I think you should use numpy array here. The second problem is the X matrix. It should be a 3D array. For example if I try with

X_train = np.random.randn(6,2,2)

then it will work.

Answer 2

The error it shows me when I run your code is

ValueError: Error when checking input: expected lstm_2_input to have 3 dimensions, but got array with shape (6, 2)

You can fix this by inputting a 3D numpy array, as user108372 mentioned

A good way to debug is this to print out model.summary(), which will give you the output shapes expected for each of the layers. Additionally, you don't always need to specify the output and input shapes. Keras will take care of that for you :) Now, a working example would be something like this:

X_train = np.random.randn(6,2,2)
y_train = pd.DataFrame( np.array([1, 2, 3, 4, 3, 4]) ).values
X_test = np.random.randn(2,2,2)
y_test = pd.DataFrame( np.array([1, 2]) ).values

model = Sequential()
model.add(LSTM(32, 
               return_sequences=False, 
               input_dim=X_train.shape[1]))
# The shape of the last Dense layer should always correspond to y_train.shape[1]
model.add(Dense(y_train.shape[1])) 
model.add(Activation("linear"))
model.compile(loss="mean_squared_error",
              optimizer="rmsprop")

model.fit(X_train, y_train)

I recommend you print out shapes of the example above and align your shapes with the shapes described here.