For example can I have Atlas, OpenBlas, MKL installed on my Ubuntu 14.04 at the same time and switch between them without recompiling Caffe?
You could also do it without changing system wide settings, for example by adding the library you want to use to the LD_PRELOAD or LD_LIBRARY_PATH environment variables. The first library on this path will be the one used to resolve the symbols against.
For example, you could run with
LD_PRELOAD=/path/to/blas/lib.so ./my_executable_using_caffe
You can see that this approach would be extremely useful as part of a benchmarking script for different implementations since it doesn't affect the benchmarking environment itself. For example (in bash):
my_libraries=/path/to/blas1.so /path/to/blas2.so
for lib in $my_libraries
do
LD_PRELOAD=${lib} ./my_executable_using_caffe
done
This approach, based on dynamic linking, applies to any other shared library your program is compiled against.