C++ linker missing library when running (SONAME behavior)

Question

I've made a program that uses two shared libraries (which I compiled) and are placed like this:

/home_directory_where_I_compile_and_run_everything
-->/lib/libjson_linux-gcc-4.4.6_libmt.so
-->/lib/libre2.so.0

When I compile my program I pass the relative location of those libraries to the linker, like this:

g++ ...... stuff ........ my_program.cc lib/libjson_linux-gcc-4.4.6_libmt.so lib/libre2.so.0

And it compiles fine, however when running the program it fails to find libre2.so, and if I inspect it with ldd, here's what happens:

....
lib/libjson_linux-gcc-4.4.6_libmt.so (0x00007f62906bc000)
libre2.so.0 => not found
....

Apparently, it does acknowledge that the path on libjson is relative, but it doesn't do that on libre2.so.0 (it trims all the path and just leaves libre2.so.0)

Can someone tell me why this happens?

Also, is there a way to modify this via a g++ argument?

Best.

* UPDATE * Whoa check this out! I've changed the name of libre2.so.0 to stuff.so, and then tried to compile essentially the same like this:

g++ ...... stuff ........ my_program.cc lib/libjson_linux-gcc-4.4.6_libmt.so lib/stuff.so

And it fails anyways; not only it does fail, it fails because it can't find "libre2.so.0".

Whyyy?

* UPDATE # 2 *

Output for readelf -d the_program.o

0x0000000000000001 (NEEDED)             Shared library: [lib/libjson_linux-gcc-4.4.6_libmt.so]
0x0000000000000001 (NEEDED)             Shared library: [libre2.so.0]

Now, if I could just make that [libre2.so.0] to be [lib/libre2.so.0] instead it would be fine.

* UPDATE # 3 *

As @troubadour found out:

When an executable is linked with a shared object which has a DT_SONAME field, then when the executable is run the dynamic linker will attempt to load the shared object specified by the DT_SONAME field rather than the using the file name given to the linker.

That's why it works with libjson.....so and not with libre2.so.0. (libjson.....so does not have an entry for SONAME).

And I finally found the exact question for what I'm looking for:

Is there any way to tell the gcc linker to ignore the SONAME entries on a shared library file and link instead to the specific file path?

Answer 1

I'll answer the second part of your question first i.e. why renaming libre2.so.0 didn't do what you expected.

The file name that you pass to the linker is irrelevant when you run the executable (unless you fail to supply -soname when building the library - see second edit below). The dependency is taken from what is called the "soname". If you run the readelf command on your library you can determine its soname eg.

readelf -d libre2.so.0 | grep SONAME

It doesn't matter if you rename the file. The result of the above command will still give you the same soname, hence the reason the program still failed to find "libre2.so.0".

As to the original part of your question it all hinges on whether the libraries have an RPATH or RUNPATH built in to them and/or what the content of your LD_LIBRARY_PATH environment variable is. These are the things the run-time linker (ld.so) will use to find the shared libraries. Try

man ld.so

for more information.

Since you built the libraries yourself you will know whether or not they used the -rpath or -runpath options at the final linking stage. Alternatively, use readelf again eg.

readelf -d libre2.so.0 | grep RPATH
readelf -d libre2.so.0 | grep RUNPATH

I suspect the above two commands will return nothing.

My guess was going to be that you have the current directory in your LD_LIBRARY_PATH which would allow the run-time linker to find lib/libjson_linux-gcc-4.4.6_libmt.so but not libre2.so.0. I notice that you've replied to my comment on your question to say that your LD_LIBRARY_PATH is empty. That's odd.

Perhaps you've somehow got the prefix "lib/" on the soname for libjson? i.e. did the readelf command for the SONAME return

lib/libjson_linux-gcc-4.4.6_libmt.so

rather than just

libjson_linux-gcc-4.4.6_libmt.so

Also, check what the program needs in terms of soname by running

readelf -d my_progam | grep NEEDED

Perhaps the "lib/" prefix is in there because of the way you passed it to gcc. If so then if you use the gcc command given by @enobayram then it will level the playing field i.e. it will fail to find libjson too.

The first thing to establish is not why it is not finding libre2.so.0 but how it is managing to find libjson. If you try running your executable from a different directory does it still work or does it now fail for libjson too? Alternatively, if you copy libre2.so.0 to be beside your executable does that change anything?

Edit

A posting on the Fedora Forum suggest that the Fedora version of ld.so has the current directory as a built-in search path. I haven't been able to verify this though but it would explain why you are picking up any libraries at all given that all the other things ld.so uses are absent on your case.

Edit 2

From the ld man page on my system

-soname=name

When creating an ELF shared object, set the internal DT_SONAME field to the specified name. When an executable is linked with a shared object which has a DT_SONAME field, then when the executable is run the dynamic linker will attempt to load the shared object specified by the DT_SONAME field rather than the using the file name given to the linker.

Copyright (c) 1991, 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009 Free Software Foundation, Inc.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back- Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License".

So, the theory in your comment is correct. If no -soname is explicitly specified when the library is built then no SONAME exists in the shared object and the NEEDED field in the executable simply has the file name given to the linker which, in your case, contained the leading "lib/".

Answer 2

When you link against shared libraries (.so), these must exist in your library load path when you execute the program. The location you provided upon link time is not "remembered" in the executable.

Alternative solutions:

• place the .so files in the same directory as the executable
• run with LD_LIBRARY_PATH=lib ./program
• install the .so files to a location in the library path, for example /usr/local/lib, before you run the program.
• link statically

Answer 3

You need tell where the shared library is in two situations.

One is at linking time.
There are several ways:

• copy library to some standard directory, like /usr/local/lib, and install it via ldconfig. In this way, no other option/operation is needed.
• use -L to tell gcc where to find the lib

One is at running time.
Also several ways:

• like linking time, install it via ldconfig, then no other action needed.

• using LD_LIBRARY_PATH.

  LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH ./a.out

• using rpath to write searching path into executable

  g++ main.cpp -lxxx -L/a/b/c -Wl,-rpath,/a/b/c

Only specifying shared library location in the linking phase, for example via -L, does not mean when you run the program the system's loader will find the needed library.

Answer 4

The correct compiler command line should be:

g++ ...... stuff ........ my_program.cc -Llib -ljson_linux-gcc-4.4.6_libmt -lre2

And you should either put the shared objects in a standard directory (like /usr/lib) or you should add their path to the LD_LIBRARY_PATH to be able to run your executable.