Computing Image Saliency via Neural Network Classifier

Question

Assume that we have a Convolutional Neural Network trained to classify (w.l.o.g. grayscale) images, in Tensor-Flow.

Given the trained net and a test image one can trace which pixels of it are salient, or "equivalently" which pixels are most responsible for the output classification of the image. A nice, explanation and implementation details in Theano, are given in this article.

Assume that for the first layer of convolutions that is directly linked with the input image, we do have the gradient for the parameters of every convolutional kernel-wrt. the classification function.

How can one propagate the gradient back to the Input layer, so to compute a partial derivative on every pixel of the image?

1. Propagating and accumulating back the gradient, would give us the salient pixels (they are those with big in-magnitude derivative).

2. To find the gradient wrt. the kernels of the first layer, so far I did:

   1. Replaced the usual loss operator with the output layer operator.
   2. Used the "compute_gradient" function,

All in all, it looks like:

• opt = tf.train.GradientDescentOptimizer(1)
• grads = opt.compute_gradients(output)
• grad_var = [(grad1) for grad in grads]
• g1 = sess.run([grad_var[0]])

Where, the "output" is the max of the output layer of the NN. And g1, is a (k, k, 1, M) tensor, since I used M: k x k convolutional kernels on the first layer.

Now, I need to find the correct way to propagate g1 on every input pixel, as to compute their derivative wrt. the output.

Answer 1

To compute the gradients, you don't need to use an optimizer, and you can directly use tf.gradients.
With this function, you can directly compute the gradient of output with respect to the image input, whereas the optimizer compute_gradients method can only compute gradients with respect to Variables.

The other advantage of tf.gradients is that you can specify the gradients of the output you want to backpropagate.

---

So here is how to get the gradients of an input image with respect to output[1, 1]:

• we have to set the output gradients to 0 everywhere except at indice [1, 1]

input = tf.ones([1, 4, 4, 1])
filter = tf.ones([3, 3, 1, 1])
output = tf.nn.conv2d(input, filter, [1, 1, 1, 1], 'SAME')

grad_output = np.zeros((1, 4, 4, 1), dtype=np.float32)
grad_output[0, 1, 1, 0] = 1.

grads = tf.gradients(output, input, grad_output)

sess = tf.Session()
print sess.run(grads[0]).reshape((4, 4))
# prints [[ 1.  1.  1.  0.]
#         [ 1.  1.  1.  0.]
#         [ 1.  1.  1.  0.]
#         [ 0.  0.  0.  0.]]