Relocation R_X86_64_32S against '_Py_NotImplementedStruct' can not be used when making a shared object; recompile with -fPIC

Question

I'm trying to install the dlib Python library. On some systems (macOS, stock Ubuntu 14.04) pip install dlib works fine, but in the Ubuntu 14.x that's part of our CircleCI environment it fails with the following error.

Linking CXX shared library dlib.so
   /usr/bin/ld: /opt/circleci/python/2.7.11/lib/libpython2.7.a(abstract.o): relocation R_X86_64_32S against '_Py_NotImplementedStruct' can not be used when making a shared object; recompile with -fPIC
   error: cmake build failed!

What could be wrong?

Answer 1

The problem was that Python needs to be compiled with the --enable-shared flag for the dlib install to succeed. While in some cases the system Python is built with this flag (e.g. on Ubuntu), the one we were using in the CI environment was installed via pyenv which doesn't set it by default.

The solution was to reinstall the pyenv-provided Python with the flag set like this:

PYTHON_CONFIGURE_OPTS="--enable-shared" pyenv install --force 2.7.11

To ensure this gets used: machine: python: version: 2.7.11 # Has to match the pyenv-installed version

Answer 2

As an FYI, my particular case was resolved by renaming '/usr/local/lib/libpython2.7.a' to '/usr/local/lib/libpython2.7.a.moved'. According to 'yum whatprovides /usr/local/lib/libpython2.7.a' output, this was not installed as a part of any packages installed via yum. Moving this out of the way in this case, solved my issue.

Here was my original error message:

  /usr/bin/ld: /usr/local/lib/libpython2.7.a(abstract.o): relocation R_X86_64_32S against `_Py_NotImplementedStruct' can not be used when making a shared object; recompile with -fPIC

/usr/local/lib/libpython2.7.a: error adding symbols: Bad value collect2: error: ld returned 1 exit status

Given that none of my installed packages accounted for the .a lib, moving it aside was an option for me.

Answer 3

I had similar problem when building matplotlib (a dependency for scikit-image) for aws lambda python 3.6 inside amazon docker container amazon-linux-python-3.6.

In short matplotlib was giving the same error as OP had for /usr/lib/libpython3.6m.a. Turned out there were two such libraries in the amazon docker container:

find / -name "libpython3.6m.a"
/usr/lib/libpython3.6m.a
/usr/lib/python3.6/config-3.6m-x86_64-linux-gnu/libpython3.6m.a

So I just renamed /usr/lib/libpython3.6m.a to something else so that matplotlib does not use it, and chooses the second option:

enter code heremv /usr/lib/libpython3.6m.a /usr/lib/libpython3.6m.a.moved

After this change, scikit-image were successful using pip3 install --no-binary scikit-image scikit-image.