Tensorflow + Keras + Convolution2d: ValueError: Filter must not be larger than the input: Filter: (5, 5) Input: (3, 350)

Question

I have been trying to run the code below which I got from here and even though I have changed almost nothing other than the image size (350,350 instead of 150, 150) is still cannot get it to work. I am getting the above filter error (in title) which I do comprehend but I am not doing it wrong so I don't understand this. It basically says that I cannot have more nodes than inputs, correct?

I was able to eventually hack my way to a solution by changing this line:

model.add(Convolution2D(32, 5, 5, border_mode='valid', input_shape=(3, IMG_WIDTH, IMG_HEIGHT)))

with this:

model.add(Convolution2D(32, 5, 5, border_mode='valid', input_shape=(IMG_WIDTH, IMG_HEIGHT, 3)))

but I would still like to understand why this worked.

Here is the code below along with the error I am getting. Would appreciate some help (I am using Python Anaconda 2.7.11).

# IMPORT LIBRARIES --------------------------------------------------------------------------------#
import glob
import tensorflow
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Convolution2D, MaxPooling2D
from keras.layers import Activation, Dropout, Flatten, Dense
from settings import RAW_DATA_ROOT

# GLOBAL VARIABLES --------------------------------------------------------------------------------#
TRAIN_PATH = RAW_DATA_ROOT + "/train/"
TEST_PATH = RAW_DATA_ROOT + "/test/"

IMG_WIDTH, IMG_HEIGHT = 350, 350

NB_TRAIN_SAMPLES = len(glob.glob(TRAIN_PATH + "*"))
NB_VALIDATION_SAMPLES = len(glob.glob(TEST_PATH + "*"))
NB_EPOCH = 50

# FUNCTIONS ---------------------------------------------------------------------------------------#
def baseline_model():
    """
    The Keras library provides wrapper classes to allow you to use neural network models developed
    with Keras in scikit-learn. The code snippet below is used to construct a simple stack of 3
    convolution layers with a ReLU activation and followed by max-pooling layers. This is very
    similar to the architectures that Yann LeCun advocated in the 1990s for image classification
    (with the exception of ReLU).
    :return: The training model.
    """
    model = Sequential()
    model.add(Convolution2D(32, 5, 5, border_mode='valid', input_shape=(3, IMG_WIDTH, IMG_HEIGHT)))
    model.add(Activation('relu'))
    model.add(MaxPooling2D(pool_size=(2, 2)))

    model.add(Convolution2D(32, 5, 5, border_mode='valid'))
    model.add(Activation('relu'))
    model.add(MaxPooling2D(pool_size=(2, 2)))

    model.add(Convolution2D(64, 5, 5, border_mode='valid'))
    model.add(Activation('relu'))
    model.add(MaxPooling2D(pool_size=(2, 2)))

    # Add a  fully connected layer layer that converts our 3D feature maps to 1D feature vectors
    model.add(Flatten())
    model.add(Dense(64))
    model.add(Activation('relu'))

    # Use a dropout layer to reduce over-fitting, by preventing a layer from seeing twice the exact
    # same pattern (works by switching off a node once in a while in different epochs...). This
    # will also serve as out output layer.
    model.add(Dropout(0.5))
    model.add(Dense(8))
    model.add(Activation('softmax'))

    # Compile model
    model.compile(loss='categorical_crossentropy',
                  optimizer='adam',
                  metrics=['accuracy'])

    return model

def train_model(model):
    """
    Simple script that uses the baseline model and returns a trained model.
    :param model: model
    :return: model
    """

    # Define the augmentation configuration we will use for training
    TRAIN_DATAGEN = ImageDataGenerator(
            rescale=1. / 255,
            shear_range=0.2,
            zoom_range=0.2,
            horizontal_flip=True)

    # Build the train generator
    TRAIN_GENERATOR = TRAIN_DATAGEN.flow_from_directory(
            TRAIN_PATH,
            target_size=(IMG_WIDTH, IMG_HEIGHT),
            batch_size=32,
            class_mode='categorical')

    TEST_DATAGEN = ImageDataGenerator(rescale=1. / 255)

    # Build the validation generator
    TEST_GENERATOR = TEST_DATAGEN.flow_from_directory(
            TEST_PATH,
            target_size=(IMG_WIDTH, IMG_HEIGHT),
            batch_size=32,
            class_mode='categorical')

    # Train model
    model.fit_generator(
            TRAIN_GENERATOR,
            samples_per_epoch=NB_TRAIN_SAMPLES,
            nb_epoch=NB_EPOCH,
            validation_data=TEST_GENERATOR,
            nb_val_samples=NB_VALIDATION_SAMPLES)

    # Always save your weights after training or during training
    model.save_weights('first_try.h5') 

# END OF FILE -------------------------------------------------------------------------------------#

and the error:

Using TensorFlow backend.
Training set: 0 files.
Test set: 0 files.
Traceback (most recent call last):
  File "/Users/christoshadjinikolis/GitHub_repos/datareplyuk/ODSC_Facial_Sentiment_Analysis/src/model/__init__.py", line 79, in <module>
    model = baseline_model()
  File "/Users/christoshadjinikolis/GitHub_repos/datareplyuk/ODSC_Facial_Sentiment_Analysis/src/model/training_module.py", line 31, in baseline_model
    model.add(Convolution2D(32, 5, 5, border_mode='valid', input_shape=(3, IMG_WIDTH, IMG_HEIGHT)))
  File "/Users/christoshadjinikolis/anaconda/lib/python2.7/site-packages/keras/models.py", line 276, in add
    layer.create_input_layer(batch_input_shape, input_dtype)
  File "/Users/christoshadjinikolis/anaconda/lib/python2.7/site-packages/keras/engine/topology.py", line 370, in create_input_layer
    self(x)
  File "/Users/christoshadjinikolis/anaconda/lib/python2.7/site-packages/keras/engine/topology.py", line 514, in __call__
    self.add_inbound_node(inbound_layers, node_indices, tensor_indices)
  File "/Users/christoshadjinikolis/anaconda/lib/python2.7/site-packages/keras/engine/topology.py", line 572, in add_inbound_node
    Node.create_node(self, inbound_layers, node_indices, tensor_indices)
  File "/Users/christoshadjinikolis/anaconda/lib/python2.7/site-packages/keras/engine/topology.py", line 149, in create_node
    output_tensors = to_list(outbound_layer.call(input_tensors[0], mask=input_masks[0]))
  File "/Users/christoshadjinikolis/anaconda/lib/python2.7/site-packages/keras/layers/convolutional.py", line 466, in call
    filter_shape=self.W_shape)
  File "/Users/christoshadjinikolis/anaconda/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py", line 1579, in conv2d
    x = tf.nn.conv2d(x, kernel, strides, padding=padding)
  File "/Users/christoshadjinikolis/anaconda/lib/python2.7/site-packages/tensorflow/python/ops/gen_nn_ops.py", line 394, in conv2d
    data_format=data_format, name=name)
  File "/Users/christoshadjinikolis/anaconda/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 703, in apply_op
    op_def=op_def)
  File "/Users/christoshadjinikolis/anaconda/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2319, in create_op
    set_shapes_for_outputs(ret)
  File "/Users/christoshadjinikolis/anaconda/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1711, in set_shapes_for_outputs
    shapes = shape_func(op)
  File "/Users/christoshadjinikolis/anaconda/lib/python2.7/site-packages/tensorflow/python/framework/common_shapes.py", line 246, in conv2d_shape
    padding)
  File "/Users/christoshadjinikolis/anaconda/lib/python2.7/site-packages/tensorflow/python/framework/common_shapes.py", line 184, in get2d_conv_output_size
    (row_stride, col_stride), padding_type)
  File "/Users/christoshadjinikolis/anaconda/lib/python2.7/site-packages/tensorflow/python/framework/common_shapes.py", line 149, in get_conv_output_size
    "Filter: %r Input: %r" % (filter_size, input_size))
ValueError: Filter must not be larger than the input: Filter: (5, 5) Input: (3, 350)

Answer 1

The problem is that the order of input_shape() changes depending the backend you are using (tensorflow or theano).

The best solution I found was defining this order in the file ~/.keras/keras.json.

Try to use the theano order with tensorflow backend, or theano order with theano backend.

Create the keras directory in your home and create the keras json: mkdir ~/.keras && touch ~/.keras/keras.json

{
    "image_dim_ordering": "th", 
    "epsilon": 1e-07, 
    "floatx": "float32", 
    "backend": "tensorflow"
}

Answer 2

Just encountered the same problem myself, when I was following a tutorial. As pointed out by @Yao Zhang, the error is caused by the order in the input_shape. There are multiple ways to solve the problem.

• Option 1: Change the order in input_shape

The line of your code

model.add(Convolution2D(32, 5, 5, border_mode='valid', input_shape=(3, IMG_WIDTH, IMG_HEIGHT)))

should be changed to

model.add(Convolution2D(32, 5, 5, border_mode='valid', input_shape=(IMG_WIDTH, IMG_HEIGHT, 3)))

which should be fine then.

• Option 2: Specify image_dim_ordering in your layers

• Option 3: Modify the keras configuration file by changing 'tf' to 'th' in your ~/.keras/keras.json