Huge spike in learning after 5000 epoch (LSTM)

Question

I\'m struggling with optimizing a LSTM NN, let me explain what I\'m trying to do :)

--> I have a data set with lets say daily temperature of my location, s