Rmpi unable to load shared libraries as non root user

Question

I'm having a problem with Rmpi wherein I try to load it and I get this error message:

> library('Rmpi')
Error in dyn.load(file, DLLpath = DLLpath, ...) :
  unable to load shared library '/usr/lib64/R/library/Rmpi/libs/Rmpi.so':
  libmpi.so.0: cannot open shared object file: No such file or directory
In addition: Warning message:
.Last.lib failed in detach() for 'Rmpi', details:
  call: dyn.unload(file.path(libpath, "libs", paste("Rmpi", .Platform$dynlib.ext,
  error: dynamic/shared library '/usr/lib64/R/library/Rmpi/libs/Rmpi.so' was not loaded
Error in library("Rmpi") : .First.lib failed for 'Rmpi'

This error does not occur when I'm logged in as root, however.

It does not appear to be a permissions issue. I checked the permissions for libmpi.so.0:

[meehan@cnl10 /]$ ll /usr/lib64/lam/lib/
total 7.4M
-rw-r--r-- 1 root root  207 May 25  2008 lam.module
-rw-r--r-- 1 root root 885K May 25  2008 liblam.a
-rw-r--r-- 1 root root 361K May 25  2008 liblamf77mpi.a
lrwxrwxrwx 1 root root   21 Apr 12  2010 liblamf77mpi.so -> liblamf77mpi.so.0.0.0
lrwxrwxrwx 1 root root   21 Apr 12  2010 liblamf77mpi.so.0 -> liblamf77mpi.so.0.0.0
-rwxr-xr-x 1 root root  73K May 25  2008 liblamf77mpi.so.0.0.0
-rw-r--r-- 1 root root 2.2M May 25  2008 liblammpi++.a
-rw-r--r-- 1 root root 509K May 25  2008 liblammpio.a
lrwxrwxrwx 1 root root   20 Apr 12  2010 liblammpi++.so -> liblammpi++.so.0.0.0
lrwxrwxrwx 1 root root   20 Apr 12  2010 liblammpi++.so.0 -> liblammpi++.so.0.0.0
-rwxr-xr-x 1 root root 167K May 25  2008 liblammpi++.so.0.0.0
lrwxrwxrwx 1 root root   15 Apr 12  2010 liblam.so -> liblam.so.0.0.0
lrwxrwxrwx 1 root root   15 Apr 12  2010 liblam.so.0 -> liblam.so.0.0.0
-rwxr-xr-x 1 root root 332K May 25  2008 liblam.so.0.0.0
-rw-r--r-- 1 root root 2.2M May 25  2008 libmpi.a
lrwxrwxrwx 1 root root   15 Apr 12  2010 libmpi.so -> libmpi.so.0.0.0
lrwxrwxrwx 1 root root   15 Apr 12  2010 libmpi.so.0 -> libmpi.so.0.0.0
-rwxr-xr-x 1 root root 655K May 25  2008 libmpi.so.0.0.0

And Rmpi.so:

[meehan@cnl10 /]$ ll /usr/lib64/R/library/Rmpi/libs/
total 108K
-rwxr-xr-x 1 root root 104K Jan 20  2011 Rmpi.so

I am running R as sudo anyway.

Relevant system info: -Linux distro: CentOS 5.5 -R version: 2.11.1 (2010-05-31) -Rmpi version: 0.5-8 -MPI implementation is openmpi

[meehan@cnl10 /]$  echo $LD_LIBRARY_PATH
/opt/lib:/opt/open-mpi/tcp-`gnu41/lib:/opt/intel/mkl/10.2/lib/em64t:/opt/intel/fce/11.1/lib:/opt/intel/cce/11.1/lib:`

Any help would be most appreciated!

Answer 1

The problem here is that OpenMPI, by default, does not register its libraries directory with the system linker. This is why some installation guides recommend you put its directories in your LD_LIBRARY_PATH variable, so that the libraries can be found at runtime. However, "adding the directories to LD_LIBRARY_PATH" must be done every time a new shell is loaded, which is why these guides suggest putting it in ~/.bashrc or the like, so that the setting is restored on every login.

However, the ~/.bashrc file (or ~/.profile, or any such) is a user-specific setting. Assuming one is logged in as root when installing openmpi, and Rmpi, and the like, which seems likely, that means that adding to these user-specific files will only set the library paths when running as root, and not as your usual runtime user.

The fix, in general, is to tell the linker where these files can be found. On my own system, which is running CentOS 7, OpenMPI 1.10.0 (using the Scientific Linux RPMs), R 3.2.3, and Rmpi 0.6-5, this is what happens when I fail to set the library path:

[dchurch@workstation ~]$ R -q -e "library('Rmpi')"
> library('Rmpi')
Error : .onLoad failed in loadNamespace() for 'Rmpi', details:
  call: dyn.load(file, DLLpath = DLLpath, ...)
  error: unable to load shared object '/usr/lib64/R/library/Rmpi/libs/Rmpi.so':
  libmpi.so.12: cannot open shared object file: No such file or directory
Error: package or namespace load failed for ‘Rmpi’
Execution halted        

If I temporarily set the linker path with a temporary variable, it works for this invocation:

[dchurch@workstation ~]$ LD_LIBRARY_PATH=/usr/lib64/openmpi/lib R -q -e "library('Rmpi')"
> library('Rmpi')       
>
>

However, to make this change permanent, the best way to do it is to register the openmpi libraries directory with the system linker itself, by creating a new file in /etc/ld.so.conf.d and running ldconfig, as so:

[dchurch@workstation ~]$ sudo sh -c 'echo /usr/lib64/openmpi/lib > /etc/ld.so.conf.d/openmpi.conf; ldconfig'
[dchurch@workstation ~]$ R -q -e "library('Rmpi')"
> library('Rmpi')
>
>

Once you have done that, Rmpi should be able to be loaded for any user, regardless of environment variables.