wrapping a C library (GSL) in a cython code by using callback

Question

I am a newbie with cython and c. I want to use cython to speed up the performance of my code. I would like to use gsl_integration library in my code for integration. update: test_gsl.pyx

cdef extern from "math.h":
    double log(double x) nogil
cdef extern from "gsl/gsl_math.h":
    ctypedef struct gsl_function:
        double (* function) (double x, void * params)
        void * params

cdef extern from "gsl/gsl_integration.h":
    ctypedef struct gsl_integration_workspace
    gsl_integration_workspace *  gsl_integration_workspace_alloc(size_t n)
    void  gsl_integration_workspace_free(gsl_integration_workspace * w)

    int  gsl_integration_qags(const gsl_function * f, double a, double b, double epsabs, double epsrel, size_t limit, gsl_integration_workspace * workspace, double *result, double *abserr)


cdef double do_callback(double x, void* params):
    return (<MyCallback>params).eval(x)

cdef class MyCallback:
    cdef double a
    def __init__(self, a):
        self.a = a
    cpdef double eval(self, double x):
        return self.a * log(x+1) * x
    def call_gsl(self):
        cdef gsl_integration_workspace* w =gsl_integration_workspace_alloc (1000)

        cdef gsl_function F
        F.function = &do_callback
        F.params = <void*>self

        cdef double result = 3, error = 5
        gsl_integration_qags(&F, 0, 1, 0, 1e-7, 1000, w, &result, &error)
        print result, error
        gsl_integration_workspace_free(w)

This .pyx code is compiled with the following setup.py file and does not raise any error message:

from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
import numpy
import sys 
ext = Extension("test_gsl", ["test_gsl.pyx"],
    include_dirs=[numpy.get_include(),
                  "/usr/include/"],
    library_dirs=["/usr/lib/"],
    libraries=["gsl"])

setup(ext_modules=[ext],
    cmdclass = {'build_ext': build_ext})

or even with the command line:

cython test_gsl.pyx
gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/vol/anaconda/lib/python2.7/site-packages -I/usr/include -I/vol/anaconda/include/python2.7 -c test_gsl.c `pkg-config --cflags gsl` 
gcc -pthread -shared test_gsl.o -L/usr/lib -L/vol/anaconda/lib -lgsl -lgslcblas -lpython2.7 `pkg-config --libs gsl` -o test_gsl.so

when it is import in python as following, it does raise errors:

>>> import pyximport; pyximport.install()
(None, <pyximport.pyximport.PyxImporter object at 0x7f0c7e888150>)
>>> import gsl
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/anaconda/lib/python2.7/site-packages/Cython-0.20.1-py2.7-linux-x86_64.egg/pyximport/pyximport.py", line 431, in load_module
    language_level=self.language_level)
  File "/anaconda/lib/python2.7/site-packages/Cython-0.20.1-py2.7-linux-x86_64.egg/pyximport/pyximport.py", line 210, in load_module
    mod = imp.load_dynamic(name, so_path)
ImportError: Building module gsl failed: ['ImportError: /users/dalek/.pyxbld/lib.linux-x86_64-2.7/gsl.so: undefined symbol: gsl_integration_qags\n']

gsl_integration_qags has been defined properly, I don't understand why I got this error again?

Answer 1

First Rule: Premature optimization is the root of all evil.

Second Rule: Follow first rule at all cost.

Third Rule: Do not use C++ complex features ( complex in comparison to C - this includes classes) if there is no need for that (even if you are a C++ fanatic like I am). This is especially true if you are mixing C++ with C libraries.

I can't see any reason why C++ classes are necessary in your example, especially because you create an unnecessary indirection (the wrapper) by doing that! If you are coding in compiled language for performance, unnecessary steps and indirections is exactly want you want to avoid! You are making your life difficult for no reason, especially because is the GSL routines in C that will do 99.9% of the computations in your program. Why not just use something like cython-gsl and resume your code to something like that (taken from cython-gsl example folder). This is much shorter, cleaner and I can't see the reason why it would not perform well given that python is not doing any heavy work (assuming that the function foo() will be converted to C which seems to be the case)!

from cython_gsl cimport *

ctypedef double * double_ptr ctypedef void * void_ptr

cdef double foo(double x, void * params) nogil:
    cdef double alpha, f
    alpha = (<double_ptr> params)[0]
    f = log(alpha*x) / sqrt(x)
    return f


def main():
    cdef gsl_integration_workspace * w
    cdef double result, error, expected, alpha
    w = gsl_integration_workspace_alloc (1000)

    expected = -4.0
    alpha = 1

    cdef gsl_function F
    F.function = &foo
    F.params = &alpha

    gsl_integration_qags (&F, 0, 1, 0, 1e-7, 1000, w, &result, &error)
    print "result          = % .18f\n" % result
    print "estimated error          = % .18f\n" % error

Answer 2

After compiling it with the below commands:

cython test_gsl.pyx
gcc -m64 -pthread -fno-strict-aliasing -Wstrict-prototypes -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/include -I/vol/dalek/anaconda/include/python2.7 -c test_gsl.c -o build/temp.linux-x86_64-2.7/test_gsl.o
gcc -pthread -shared -L/usr/lib/ -L/vol/dalek/anaconda/lib -o test_gsl.so  build/temp.linux-x86_64-2.7/test_gsl.o -lpython2.7  -lgsl -lgslcblas -lm

If I import test_gsl in python without pyximport, it works perfectly. Since pyximport doesn't support linking against external libraries.