How to understand the Cifar10 prediction output?

问题

I have trained Cifar10(caffe) model for two classes classification. Pedestrian and non-pedestrian. Training looks fine, I have updated weights in a caffemodel file. I used two labels 1 for pedestrians and 2 for non-pedestrians, together with images for pedestrians (64 x 160) and background images (64 x 160). After training, I do testing with positive image(pedestrian image) and negative image (background image). My testing prototxt file is as shown below

name: "CIFAR10_quick_test"
layers 
{
  name: "data"
  type: MEMORY_DATA
  top: "data"
  top: "label"
  memory_data_param 
  {
    batch_size: 1
    channels: 3
    height: 160
    width: 64
  }
  transform_param 
  {
    crop_size: 64
    mirror: false
    mean_file: "../../examples/cifar10/mean.binaryproto"
  }
}
layers {
  name: "conv1"
  type: CONVOLUTION
  bottom: "data"
  top: "conv1"
  blobs_lr: 1
  blobs_lr: 2
  convolution_param {
    num_output: 32
    pad: 2
    kernel_size: 5
    stride: 1
  }
}
layers {
  name: "pool1"
  type: POOLING
  bottom: "conv1"
  top: "pool1"
  pooling_param {
    pool: MAX
    kernel_size: 3
    stride: 2
  }
}
layers {
  name: "relu1"
  type: RELU
  bottom: "pool1"
  top: "pool1"
}
layers {
  name: "conv2"
  type: CONVOLUTION
  bottom: "pool1"
  top: "conv2"
  blobs_lr: 1
  blobs_lr: 2
  convolution_param {
    num_output: 32
    pad: 2
    kernel_size: 5
    stride: 1
  }
}
layers {
  name: "relu2"
  type: RELU
  bottom: "conv2"
  top: "conv2"
}
layers {
  name: "pool2"
  type: POOLING
  bottom: "conv2"
  top: "pool2"
  pooling_param {
    pool: AVE
    kernel_size: 3
    stride: 2
  }
}
layers {
  name: "conv3"
  type: CONVOLUTION
  bottom: "pool2"
  top: "conv3"
  blobs_lr: 1
  blobs_lr: 2
  convolution_param {
    num_output: 64
    pad: 2
    kernel_size: 5
    stride: 1
  }
}
layers {
  name: "relu3"
  type: RELU
  bottom: "conv3"
  top: "conv3"
}
layers {
  name: "pool3"
  type: POOLING
  bottom: "conv3"
  top: "pool3"
  pooling_param {
    pool: AVE
    kernel_size: 3
    stride: 2
  }
}
layers {
  name: "ip1"
  type: INNER_PRODUCT
  bottom: "pool3"
  top: "ip1"
  blobs_lr: 1
  blobs_lr: 2
  inner_product_param {
    num_output: 64
  }
}
layers {
  name: "ip2"
  type: INNER_PRODUCT
  bottom: "ip1"
  top: "ip2"
  blobs_lr: 1
  blobs_lr: 2
  inner_product_param {
    num_output: 10
  }
}
layers {
  name: "prob"
  type: SOFTMAX
  bottom: "ip2"
  top: "prob"
}

For testing, I used test_predict_imagenet.cpp and did some modifications especially for paths and image size.

I can't figure out the test output. When I test with positive image, I got the output as

I0813 01:55:30.378114  7668 test_predict_cifarnet.cpp:72] 1
I0813 01:55:30.379082  7668 test_predict_cifarnet.cpp:72] 3.90971e-007
I0813 01:55:30.381088  7668 test_predict_cifarnet.cpp:72] 0.00406029
I0813 01:55:30.383090  7668 test_predict_cifarnet.cpp:72] 0.995887
I0813 01:55:30.384119  7668 test_predict_cifarnet.cpp:72] 1.96203e-006
I0813 01:55:30.385095  7668 test_predict_cifarnet.cpp:72] 3.50333e-005
I0813 01:55:30.386119  7668 test_predict_cifarnet.cpp:72] 1.2796e-008
I0813 01:55:30.387097  7668 test_predict_cifarnet.cpp:72] 1.48836e-005
I0813 01:55:30.389093  7668 test_predict_cifarnet.cpp:72] 1.12237e-007
I0813 01:55:30.390100  7668 test_predict_cifarnet.cpp:72] 4.71238e-008
I0813 01:55:30.391101  7668 test_predict_cifarnet.cpp:72] 9.04134e-008

When I test with a negative image, I got the output as

I0813 01:53:40.896139 10856 test_predict_cifarnet.cpp:72] 1
I0813 01:53:40.897117 10856 test_predict_cifarnet.cpp:72] 6.20882e-006
I0813 01:53:40.898115 10856 test_predict_cifarnet.cpp:72] 7.10468e-005
I0813 01:53:40.900184 10856 test_predict_cifarnet.cpp:72] 0.999911
I0813 01:53:40.901185 10856 test_predict_cifarnet.cpp:72] 3.4275e-006
I0813 01:53:40.902189 10856 test_predict_cifarnet.cpp:72] 2.38526e-007
I0813 01:53:40.903192 10856 test_predict_cifarnet.cpp:72] 2.29073e-007
I0813 01:53:40.905187 10856 test_predict_cifarnet.cpp:72] 1.7243e-006
I0813 01:53:40.906188 10856 test_predict_cifarnet.cpp:72] 5.40765e-007
I0813 01:53:40.908195 10856 test_predict_cifarnet.cpp:72] 1.57534e-006
I0813 01:53:40.909195 10856 test_predict_cifarnet.cpp:72] 3.72312e-006

How to understand the testing output?

Is there any more effecient testing algorithm for testing the model from video feed (frame by frame from video clip)?

回答1:

Why do you have num_output: 10 for the last layer ip2? you only need 2-way classifier? Why are you using labels 1 and 2 instead of 0 and 1?

What you got: You have 11 outputs: one is the "label" output from the data layer, and the other 10 outputs are the 10-vector output of the softmax layer. It is unclear what the values of the 10-vector are since you only trained using two labels, thus 8 out of 10 entries were not supervised at all. Moreover, judging by the first output it seems both tests were samples with label 1 and not 2.

What you should do:
1. Change the topmost fully connected layer to have only two outputs (I also changed the format to match the new version protobuff)

layer {
  name: "ip2/pedestrains"
  type: "InnerProduct"
  bottom: "ip1"
  top: "ip2"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  inner_product_param {
    num_output: 2 # This is what you need changing
  }
}

2. Change the binary labels in your training data to 0/1 rather that 1/2.

Now you can train again and see what you get.

来源：https://stackoverflow.com/questions/31972448/how-to-understand-the-cifar10-prediction-output

标签

c++

deep-learning

caffe

conv-neural-network