perform the exact same convolution as in theano's conv2d

Question

I have an existing classification model that was trained using theano's conv2d under theano.tensor.nnet. Now I have to use this model to do some sort of prediction in Java.

I implement a simple convolution in Python(In the end, I will code it in Java) as per some documentation(https://developer.apple.com/Library/ios/documentation/Performance/Conceptual/vImage/ConvolutionOperations/ConvolutionOperations.html). For example, for a 2*2 kernel (k11,k12,k21,k22), one of the areas under the kernel is (a11,a12,a21,a22). The convolution is performed by a11*k11 + a12*k12 + a21*k21 + a22*k22.

Unfortunately, as I test my convolution code and theano's conv code with some dummy matrix and kernels, they give different results. Only in some rare cases, they give same results.

It seems to me that there are many variants of convolution algorithm and I have to implement the exact same convolution algorithm as used by theano's ConvOp. However, I can't find any material describing theano's Conv2d algorithm.

Could you explain a little bit about theano's conv2d algorithm?

The following is my python code for convolution:

def convolution(img_arr, kernel):
    h, w = img_arr.shape
    k = len(kernel)
    k_flat = kernel.flatten()
    neww = w - len(kernel) + 1
    newh = h - len(kernel) + 1
    after = np.zeros((newh, neww))
    for i in range(neww):
        for j in range(newh):
            after[j,i] = (img_arr[j:j+k,i:i+k].flatten()*k_flat).sum()
    return after

Answer 1

Theano convolution does exactly the same thing as scipy.signal.convolve2d. This is exploited/tested e.g. here. For self-containedness, try copy+pasting:

import numpy as np
from scipy.signal import convolve2d
import theano
import theano.tensor as T

rng = np.random.RandomState(42)
image = rng.randn(500, 500).astype(np.float32)
conv_filter = rng.randn(32, 32).astype(np.float32)

img = T.tensor4()
fil = T.tensor4()

for border_mode in ["full", "valid"]:
    scipy_convolved = convolve2d(image, conv_filter, mode=border_mode)
    theano_convolve2d = theano.function([img, fil], T.nnet.conv2d(img, fil,
                                        border_mode=border_mode))
    theano_convolved = theano_convolve2d(image.reshape(1, 1, 500, 500),
                                         conv_filter.reshape(1, 1, 32, 32))
    l2_discrepancy = np.sqrt(((scipy_convolved - theano_convolved) ** 2).sum())
    print "Discrepancy %1.5e for border mode %s" % (l2_discrepancy, border_mode)
    print "Norms of convolutions: %1.5e, %1.5e" % (
        np.linalg.norm(scipy_convolved.ravel()),
        np.linalg.norm(theano_convolved.ravel()))

It outputs

Discrepancy 9.42469e-03 for border mode full
Norms of convolutions: 1.65433e+04, 1.65440e+04
Discrepancy 9.03687e-03 for border mode valid
Norms of convolutions: 1.55051e+04, 1.55054e+04

Since scipy implements a standard convolution, so does theano.

Answer 2

Actually you and the (Theano, Scripy) are all right. the reason is that: your are using the different convolution2D. the Theano and Script using the convolution2D defined in Math that should rotate the kernel. but you did not (ref: http://www.songho.ca/dsp/convolution/convolution.html#convolution_2d). SO if your kernel is like this:

[1, 2, 3]
[4, 5, 6]
[7, 8, 9]

than change it to (center symmetry):

[9, 8, 7]
[6, 5, 4]
[3, 2, 1]

So using your method will get the same answer as Theano/Scripy