Numpy import fails on multiarray extension library when called from embedded Python within a C++ application

Question

I\'m running a C++ application which tries to run python using the https://docs.python.org/3.5/extending/embedding.html function calls. This is the error that the applicatio

Answer 1

Root Cause

This error occurs because multiarray.cpython-35m-x86_64-linux-gnu.so module in numpy depends on libpythonx.x.so, be it is not explicit link the libpythonx.x.so. So if you use ldd -d multiarray.cpython-35m-x86_64-linux-gnu.so you will not see the python in the list.

Python doesn't have issue because python binary depends on libpython.x.x.so, so when numpy load multiarray.cpython-35m-x86_64-linux-gnu.so by using dlopen. libdl.so will try to resolve the undefined symbols by checking the dependent shared library of the main program which is python. It will find it in libpython.x.x.so.

Solution

After knowing the root cause the solution is very easy, just help libdl.so to be able to find libpython.x.x.so. There are at least two way to achieve that:

1. Use dlopen("libpythonx.x.so", RTLD_GLOBAL). After open this so use RTLD_GLOBAL flag, it make symbol in libpythonx.x.so available for symbol resolution of subsequently loaded shared objects.
2. In main program which embed python, add the libpythonx.x.so into its dependency library.