Cython wrapping a class that uses another library

问题

I've got some C++ code dbscan.cpp and dbscan.h that work great standalone. Now I'm trying to wrap it in Cython. I'm not sure how to do this correctly, and I'm impeded by limited knowledge about compilers and linkers and libraries and makefiles.

Here's PyDBSCAN_lib.pyx:

# distutils: language = c++ 
# distutils: sources = dbscan.cpp

from libcpp.vector cimport vector
from libcpp.string cimport string
from libcpp cimport bool

cdef extern from "dbscan.h":
    cdef cppclass DBSCAN:
        #DBSCAN(int minPts, int eps) except +
        DBSCAN(int minPts) except +
        void start()
        void findNeighbors(int pid, vector[int]& neighbors)
        void readFile(string filename, bool lastColIsTrueCluster)
        void buildDistMatrix()
        void calcEps()
        void calcNumNeighbors()
        void initLabels()
        void writeFile(string filename)

cdef class PyDBSCAN:
    cdef DBSCAN *thisptr
    def __cinit__(self, int minPts):
        self.thisptr = new DBSCAN(minPts)
    def __dealloc__(self):
        del self.thisptr
    def start(self):
        self.thisptr.start()
    def findNeighbors(self, int pid, vector[int]& neighbors):
        self.thisptr.findNeighbors(pid, neighbors)
    def readFile(self, string filename, bool lastColIsTrueCluster):
        self.thisptr.readFile(filename, lastColIsTrueCluster)
    def buildDistMatrix(self):
        self.thisptr.buildDistMatrix()
    def calcEps(self):
        self.thisptr.calcEps()
    def calcNumNeighbors(self):
        self.thisptr.calcNumNeighbors()
    def initLabels(self):
        self.thisptr.initLabels()
    def writeFile(self, string filename):
        self.thisptr.writeFile(filename)

As you can see the top half makes reference to my c++ code, and the bottom is a wrapper class.

And here's the setup.py, which I understand is sort of like a makefile:

from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
#from Cython.Build import cythonize
import os

os.environ['CC'] = '/app/gcc/4.8.2/bin/g++ -std=c++11'
os.environ['CXX'] = '/app/gcc/4.8.2/bin/g++ -std=c++11'
os.environ['CPP'] = '/app/gcc/4.8.2/bin/g++ -std=c++11'
os.environ['CMAKE_CXX_COMPILER'] = '/app/gcc/4.8.2/bin/g++ -std=c++11'

modules = [Extension("PyDBSCAN_lib", 
                sources=["PyDBSCAN_lib.pyx"],
                include_dirs = [".", "/usr/local/elemental/0.81/HybridRelease/include"],
                libraries = ["mpi_cxx", "mpi", "m", "elemental"],
                library_dirs = ["/usr/local/lib", "/usr/lib", "/usr/local/elemental/0.81/HybridRelease/lib"],
                language = "c++")]

setup(ext_modules = modules, cmdclass = {"build_ext" : build_ext})

I'm trying to generate a PyDBSCAN_lib.so so that I can import it in any regular python script.

I'm problem is that dbscan.cpp makes use of some types in the Elemental library, and I can't find the right configuration to specify this in setup.py. Currently it generates this:

/usr/bin/ld: /usr/local/elemental/0.81/HybridRelease/lib/libelemental.a(matrix.cpp.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/usr/local/elemental/0.81/HybridRelease/lib/libelemental.a: could not read symbols: Bad value
collect2: error: ld returned 1 exit status

edit: for the record, here's how I'm compiling it with g++

include /usr/local/elemental/0.81/HybridRelease/conf/elemvariables 

db: dbscan_main.cpp dbscan.cpp 

    ${CXX} ${ELEM_COMPILE_FLAGS} -fopenmp $^ -o $@ ${ELEM_LINK_FLAGS} ${ELEM_LIBS}

where elemvariables contains various compile options, but -fPIC is not among them.

I'd appreciate any help on this. Thanks.

回答1:

Have you tried doing what the linker ld error is recommending? You can pass the -fPIC flag to the compiler in the Extension objection construction in setup.py:

Extension("PyDBSCAN_lib",
          # ...
          extra_compile_args=['-fPIC'],
          extra_link_args=['-fPIC']
          # ...
          )

Not sure if this should be a compiler or linker flag; I am mentioning both possibilities so that you are aware of both.

回答2:

You need to compile elemental using the -fPIC compiler flag in order to use it from a shared object such as a Python extension module. Linking a normal executable to doesn't have this requirement; this is one of the requirements for code that's part a shared object.

Distutils should automatically use the -fPIC flag on the code that it's compiling, since it knows that it's building a shared object.

来源：https://stackoverflow.com/questions/22971872/cython-wrapping-a-class-that-uses-another-library

标签

python

c++

cython