Caffe/pyCaffe: set all GPUs

Question

is possible to set all GPUs for Caffe (especially pyCaffe)?

Something like:

caffe train -solver examples/mnist/lenet_solver.prototxt -gpu all

Answer 1

AFAIK Caffe is not supporting multi gpu training at the moment. It is planned for future release. See a discussion here.
It seems like NVIDIA's branch of caffe has this functionality. See the issue here.

Answer 2

Both forks have supported multi-GPU for a while now.

• BVLC/caffe got support for multi-GPU on 08/13/2015 (see commit, issue).
• NVIDIA/caffe got support for multi-GPU on 06/19/2015 (see release note).

You may be interested to know that there is a pretty serious outstanding issue with multi-GPU and PythonLayers (see issue, temporary fix).

Answer 3

It seems that Caffe now starts supporting training on multi-GPUs for C++ interface. See the docs on CommandLine interface.

# train on GPUs 0 & 1 (doubling the batch size)
caffe train -solver examples/mnist/lenet_solver.prototxt -gpu 0,1
# train on all GPUs (multiplying batch size by number of devices)
caffe train -solver examples/mnist/lenet_solver.prototxt -gpu all