Because the word "layer" often means different things when applied to a convolutional layer (some treat everything up through pooling as a single layer, others treat convolution, nonlinearity, and pooling as separate "layers"; see fig 9.7) it's not clear to me where to apply dropout in a convolutional layer.
Does dropout happen between nonlinearity and pooling?
E.g., in TensorFlow would it be something like:
kernel_logits = tf.nn.conv2d(input_tensor, ...) + biases activations = tf.nn.relu(kernel_logits) kept_activations = tf.nn.dropout(activations, keep_prob) output = pool_fn(kept_activations, ...)
You could probably try applying dropout at different places, but in terms of preventing overfitting not sure you're going to see much of a problem before pooling. What I've seen for CNN is that
tensorflow.nn.dropout gets applied AFTER non-linearity and pooling:
# Create a convolution + maxpool layer for each filter size pooled_outputs =  for i, filter_size in enumerate(filters): with tf.name_scope("conv-maxpool-%s" % filter_size): # Convolution Layer filter_shape = [filter_size, embedding_size, 1, num_filters] W = tf.Variable(tf.truncated_normal(filter_shape, stddev=0.1), name="W") b = tf.Variable(tf.constant(0.1, shape=[num_filters]), name="b") conv = tf.nn.conv2d( self.embedded_chars_expanded, W, strides=[1, 1, 1, 1], padding="VALID", name="conv") # Apply nonlinearity h = tf.nn.relu(tf.nn.bias_add(conv, b), name="relu") # Maxpooling over the outputs pooled = tf.nn.max_pool( h, ksize=[1, sequence_length - filter_size + 1, 1, 1], strides=[1, 1, 1, 1], padding='VALID', name="pool") pooled_outputs.append(pooled) # Combine all the pooled features num_filters_total = num_filters * len(filters) self.h_pool = tf.concat(3, pooled_outputs) self.h_pool_flat = tf.reshape(self.h_pool, [-1, num_filters_total]) # Add dropout with tf.name_scope("dropout"): self.h_drop = tf.nn.dropout(self.h_pool_flat, self.dropout_keep_prob)