In Caffe we can define two learning rates for each layer: one for weights and one for the bias:
layer { name: "conv1" type: "Convolution&qu