Caffe accuracy bigger than 100%

Question

I'm building one but, and when I use the custom train function provided on lenet example with a batch size bigger than 110 my accuracy gets bigger than 1 (100%).

If I use batch size 32, I get 30 percent of accuracy. Batch size equal 64 my net accuracy is 64. And batch size equal to 128, the accuracy is 1.2.

My images are 32x32. Train dataset: 56 images of Neutral faces. 60 images of Surprise faces. Test dataset: 15 images of Neutral faces. 15 images of Surprise faces.

This is my code:

def train(solver):

niter = 200
test_interval = 25 

train_loss = zeros(niter)
test_acc = zeros(int(np.ceil(niter / test_interval)))
output = zeros((niter, 32, 2))

for it in range(niter):
    solver.step(1)
    train_loss[it] = solver.net.blobs['loss'].data
    solver.test_nets[0].forward(start='conv1')
    output[it] = solver.test_nets[0].blobs['ip2'].data[:32]
    if it % test_interval == 0:
        print 'Iteration', it, 'testing...'

        correct = 0

        for test_it in range(100):
            solver.test_nets[0].forward()
            correct += sum(solver.test_nets[0].blobs['ip2'].data.argmax(1) == solver.test_nets[0].blobs['label'].data)

        test_acc[it // test_interval] = correct / 1e4

So, what is wrong with my code?

Answer 1

In your testing code you run 100 iterations (for test_it in range(100)), on each iteration you compute correct as number of examples in a batch that are correct. You then divide that number by 1e4.

Let's assume your model is very good and has almost 100% prediction rate. Then with batch size of 32 on each of 100 iterations you will add 32 to correct, yielding 3200. You then divide it by 1e4, ending up with 0.32, which is almost consistent with what you see (your number is slightly less because sometimes your model does mispredict the target).

To fix it, you can replace

test_acc[it // test_interval] = correct / 1e4

with

test_acc[it // test_interval] = correct / (100.0 * batch_size)