use caffe to train Lenet with CSV data

Question

Excuse me, I have a question on using caffe for hd data? I try to run an example on the Kaggle mnist csv data with the following steps

1. use h5py

Answer 1

I assume you have one hdf5 data file 'data/mnist_train_h5.hd5'.

1. As you can see from the error message you got, "HDF5Data" layer does not support data transformation. Specifically, you cannot scale the data by the layer.
   Thus, any transformations you wish to have, you must apply them yourself during the creation of 'data/mnist_train_h5.hd5'.

2. "HDF5Data" layer does not accept data_param, but rather hdf5_data_param with a parameter source specifying a list of hd5 binary files. In your case you should prepare an extra text file 'data/mnist_train_h5.txt' with a single line:

data/mnist_train_h5.hd5

This text file will tell caffe to read 'data/mnist_train_h5.hd5'.

The resulting layer should look like:

layer {
  name: "mnist"
  type: "HDF5Data"
  top: "data"
  top: "label"
  hdf5_data_param {
    source: "data/mnist_train_h5.txt"
    batch_size: 64
  }
  include {
    phase: TRAIN
  }
}