MPI - error loading shared libraries

Question

The problem I faced has been solved here: Loading shared library in open-mpi/ mpi-run

I know not how, setting LD_LIBRARY_PATH or specifying -x LD_LIBRARY_PATH fixes the problem, when my installation itself specifies the necessary -L arguments. My installation is in ~/mpi/

I have also included my compile-link configs.

$ mpic++ -showme:version 
mpic++: Open MPI 1.6.3 (Language: C++)

$ mpic++ -showme
g++ -I/home/vigneshwaren/mpi/include -pthread -L/home/vigneshwaren/mpi/lib
-lmpi_cxx -lmpi -ldl -lm -Wl,--export-dynamic -lrt -lnsl -lutil -lm -ldl

$ mpic++ -showme:libdirs
/home/vigneshwaren/mpi/lib

$ mpic++ -showme:libs
mpi_cxx mpi dl m rt nsl util m dl    % Notice mpi_cxx here %

When I compiled with mpic++ <file> and ran with mpirun a.out I got a (shared library) linker error

error while loading shared libraries: libmpi_cxx.so.1: 
cannot open shared object file: No such file or directory

The error has been fixed by setting LD_LIBRARY_PATH. The question is how and why? What am i missing? Why is LD_LIBRARY_PATH required when my installation looks just fine.

Answer 1

libdl, libm, librt, libnsl and libutil are all essential system-wide libraries and they come as part of the very basic OS installation. libmpi and libmpi_cxx are part of the Open MPI installation and in your case are located in a non-standard location that must be explicitly included in the linker search path LD_LIBRARY_PATH.

It is possible to modify the configuration of the Open MPI compiler wrappers and make them pass the -rpath option to the linker. -rpath takes a library path and appends its to a list, stored inside the executable file, which tells the runtime link editor (a.k.a. the dynamic linker) where to search for libraries before it consults the LD_LIBRARY_PATH variable. For example, in your case the following option would suffice:

-Wl,-rpath,/home/vigneshwaren/mpi/lib

This would embed the path to the Open MPI libraries inside the executable and it would not matter if that path is part of LD_LIBRARY_PATH at run time or not.

To make the corresponding wrapper add that option to the list of compiler flags, you would have to modify the mpiXX-wrapper-data.txt file (where XX is cc, c++, CC, f90, etc.), located in mpi/share/openmpi/. For example, to make mpicc pass the option, you would have to modify /home/vigneshwaren/mpi/share/openmpi/mpicc-wrapper-data.txt and add the following to the line that starts with linker_flags=:

linker_flags= ... -Wl,-rpath,${prefix}/lib

${prefix} is automatically expanded by the wrapper to the current Open MPI installation path.

Answer 2

In my case, I just simply appends

export LD_LIBRARY_PATH=/PATH_TO_openmpi-version/lib:$LD_LIBRARY_PATH

For example

export LD_LIBRARY_PATH=/usr/local/openmpi-1.8.1/lib:$LD_LIBRARY_PATH

into $HOME/.bashrc file and then source it to active again source $HOME/.bashrc.

Answer 3

I installed mpich 3.2 using the following command on Ubuntu.

sudo apt-get install mpich

When I tried to run the mpi process using mpiexec, I got the same error.

/home/node1/examples/.libs/lt-cpi: error while loading shared libraries: libmpi.so.0: cannot open shared object file: No such file or directory

Configuring LD_LIBRARY_PATH didn't fix my problem.

I did a search for the file 'libmpi.so.0' on my machine but couldn't find it. Took me some time to figure out that 'libmpi.so.0' file is named as 'libmpi.so' on my machine. So I renamed it to 'libmpi.so.0'.

It solved my problem!

If you are having the same problem and you installed the library through apt-get, then do the following.

The file 'libmpi.so' should be in the location '/usr/lib/'. Rename the file to 'libmpi.so.0'

mv /usr/lib/libmpi.so /usr/lib/libmpi.so.0

After that MPI jobs should run without any problem.

If 'libmpi.so' is not found in '/usr/lib', you can get its location using the following command.

whereis libmpi.so

Answer 4

first, run this command

$ sudo apt-get install libcr-dev

if still have this problem then configure LD_LIBRARY_PATH like this:

export LD_LIBRARY_PATH=/usr/local/mpich-3.2.1/lib:$LD_LIBRARY_PATH

then add it to ~/.bashrc before this line:

[ -z "$PS1" ] && return

Answer 5

Simply running

$ ldconfig

appears to me as a better way to solve the problem (taken from a comment on this question). In particular, since it avoids misuse of the LD_LIBRARY_PATH environment variable. See here and here, for why I believe it's misused to solve the problem at hand.