Prepare C-based Cython package to publish on pypi

问题

I'm going to create C library and I would like to create Python wrapper using Cython.

Right now I have mylib.a file compiled and bundled (C files) and I want to wrap methods from my library in Cython.

I successfully created .pyx and .pxd files and I can build it using python setup.py build_ext command. My problem appears when I try publish it on pypi. If I run my setup.py, create wheel and publish it I can download it from pypi - but I can't run import mylib.

I read a lot online tutorials for this. Few of them shown that theirs C code compiled on user side. My codebase will be constructed from many files and I would prefer to supply already build C code as .a file.

My file structure (only important files):

/ lib
    - mylib.a
    - *.h files (for mylib.a)
- setup.py
/ mylib
    - mylib.pyx
    - cmylib.pxd
    - __init__.py

My __init__.py file (after importing package downloaded from pip it throws errors here):

from . import mylib

My setup.py (only important parameters - in my opinion):

setup(name='mylib'
      packages=['mylib'],
      ext_modules = [Extension(
          name="mylib", 
          sources=["mylib/mylib.bycython.c"], 
          include_dirs = [numpy.get_include(), "lib/"],
          extra_objects=["lib/mylib.a"])],
        "build_ext": build_ext
      }
)

(I build mylib.bycode.c before python setup.py build_ext using cython from mylib.pyx. According to this article it would make installing package faster and would not require user to have same cython version.)

It maybe worth mentioning - after building my package I get .so file for mylib. If I copy it to /mylib directory then from parent directory I can import mylib and access my methods using mylib.mylib.say_hello(). However it doesn't work on package installed from pip (on other PC) neither I don't want to use mylib.mylib.

If I should provide more info - let me know.

Edit:

My project (I obfuscated name earlier, so here goes real project) on GitHub: https://github.com/franiis/statr-python

I want to successfully run say_hello() method from statr.pyx (other methods probably will not work).

I know code has some problems, but I want to first have working core to fix and update everything. To build project use build_script.sh. upload_script.sh creates wheel and publishes it.

回答1:

There are several issues in the setup.py.

The linked to library is not being compiled by setup.py. This means it must be compiled manually.
The linked to library is a pre-compiled statically linked archive. This is not portable and cannot be distributed to anything other than the exact versions of GCC/glibc used to build it.
The cython extension code is not being built by setup.py. Changes to .pyx/.pxd will not be reflected in the package.

Try something like:

from setuptools import setup, find_packages, Extension
from Cython.Build import cythonize, build_ext

extensions = [Extension(
          name="statr._ext", 
          sources=["statr/_ext.pyx", "lib/mylib.c"],
          depends="lib/mylib.h",
          include_dirs=[numpy.get_include(), "lib/"]
]

setup(name='statr'
      packages=find_packages(),
      ext_modules=cythonize(extensions),
      build_ext: build_ext
      }
)

With the above, the name of the extension will be statr._ext. The name of the pyx should be _ext.pyx.

mylib.c will be built and linked to the extension by setup.py. Any additional compiler directives required by mylib.c should be added to the extension.

The package name will be statr. You should import anything from _ext that you want to be available from the top level statrd module in its __init__.py, for example

from ._ext import my_cython_function

来源：https://stackoverflow.com/questions/49471084/prepare-c-based-cython-package-to-publish-on-pypi

标签

python

cython

pypi