Using Keras for video prediction (time series)

Question

I want to predict the next frame of a (greyscale) video given N previous frames - using CNNs or RNNs in Keras. Most tutorials and other information regarding ti

Answer 1

After doing lots of research, I finally stumbled upon the Keras Example for the ConvLSTM2D layer (Already mentioned by Marcin Możejko), which does exactly what I need.

In the current version of Keras (v1.2.2), this layer is already included and can be imported using

from keras.layers.convolutional_recurrent import ConvLSTM2D 

To use this layer, the video data has to be formatted as follows:

[nb_samples, nb_frames, width, height, channels] # if using dim_ordering = 'tf'
[nb_samples, nb_frames, channels, width, height] # if using dim_ordering = 'th'