LSTM Sequence Prediction in Keras just outputs last step in the input

Question

I am currently working with Keras using Tensorflow as the backend. I have a LSTM Sequence Prediction model shown below that I am using to predict one step ahead in a data series (input 30 steps [each with 4 features], output predicted step 31).

model = Sequential()

model.add(LSTM(
    input_dim=4,
    output_dim=75,
    return_sequences=True))
model.add(Dropout(0.2))

model.add(LSTM(
    150,
    return_sequences=False))
model.add(Dropout(0.2))

model.add(Dense(
    output_dim=4))
model.add(Activation("linear"))

model.compile(loss="mse", optimizer="rmsprop")
return model

The issue I'm having is that after training the model and testing it - even with the same data it trained on - what it outputs is essentially the 30th step in the input. My first thought is the patterns of my data must be too complex to accurately predict, at least with this relatively simple model, so the best answer it can return is essentially the last element of the input. To limit the possibility of over-fitting I've tried turning training epochs down to 1 but the same behavior appears. I've never observed this behavior before though and I have worked with this type of data before with successful results (for context, I'm using vibration data taken from 4 points on a complex physical system that has active stabilizers; the prediction is used in a pid loop for stabilization hence why, at least for now, I'm using a simpler model to keep things fast).

Does that sound like the most likely cause, or does anyone have another idea? Has anyone seen this behavior before? In case it helps with visualization here is what the prediction looks like for one vibration point compared to the desired output (note, these screenshots are zoomed in smaller selections of a very large dataset - as @MarcinMożejko noticed I did not zoom quite the same both times so any offset between the images is due to that, the intent is to show the horizontal offset between the prediction and true data within each image):

...and compared to the 30th step of the input:

Note: Each data point seen by the Keras model is an average over many actual measurements with the window of the average processed along in time. This is done because the vibration data is extremely chaotic at the smallest resolution I can measure so instead I use this moving average technique to predict the larger movements (which are the more important ones to counteract anyway). That is why the offset in the first image appears as many points off instead of just one, it is 'one average' or 100 individual points of offset. .

-----Edit 1, code used to get from the input datasets 'X_test, y_test' to the plots shown above-----

model_1 = lstm.build_model()  # The function above, pulled from another file 'lstm'

model_1.fit(
    X_test,
    Y_test,
    nb_epoch=1)

prediction = model_1.predict(X_test)

temp_predicted_sensor_b = (prediction[:, 0] + 1) * X_b_orig[:, 0]

sensor_b_y = (Y_test[:, 0] + 1) * X_b_orig[:, 0]

plot_results(temp_predicted_sensor_b, sensor_b_y)
plot_results(temp_predicted_sensor_b, X_b_orig[:, 29])

For context:

X_test.shape = (41541, 30, 4)

Y_test.shape = (41541, 4)

X_b_orig is the raw (averaged as described above) data from the b sensor. This is multiplied by the prediction and input data when plotting to undo normalization I do to improve the prediction. It has shape (41541, 30).

----Edit 2----

Here is a link to a complete project setup to demonstrate this behavior:

https://github.com/ebirck/lstm_sequence_prediction

Answer 1

That is because for your data(stock data?), the best prediction for 31st value is the 30th value itself.The model is correct and fits the data. I also have similar experience predicting the stock data.