Python building cython extension with setup creates subfolder when __init__.py exists

Question

I am trying to compile a simple cython module using the following setup.py:

from distutils.core import setup
from Cython.Build import cythonize

setup(
    ext_modules=cythonize("verifier_c.pyx"),
)

I have the following folder structure:

.
c_ext/
  __init__.py
  verifier_c.pyx
  setup.py

If I run the following:

python setup.py build_ext --inplace

I get an extra c_ext subfolder like this:

.
c_ext/
  build/
    ...
  c_ext/
    verifier_c.so
  __init__.py
  verifier_c.pyx
  setup.py

But if I remove the __init__.py file, I get the verifier_c.so file in the same folder as verifier_c.pyx.

I did not find where this behavior is documented, but I would like to keep verifier_c.so in the same folder as verifier_c.pyx but without having to delete __init__.py each time I run setup.py. How can I achieve that?

Answer 1

As mentioned in the comments, the setup.py should not live inside your package. As far as I know the build_ext commands has no option (apart from --inplace) to specify a target path. You can find some documentation here. Also this question deals with a similar topic.

To adapts the required package structure your package would have to look like:

c_ext/
    setup.py
    myfile.py
    verifier/
        __init__.py
        verifier_c.pyx

You will get an extension that lives in the verifier package:

me@machine:~/c_ext/$ python setup.py build_ext --inplace

c_ext/
    setup.py
    myfile.py
    verifier/
        __init__.py
        verifier_c.pyx
        verifier_c.so

You can then import verifier_c from the verifier package. For example from myfile.py this would look like:

from verifier import verifier_c
...

You can manage a separate package (and folder) for each Cython extension or create one sub folder that contains all of them. You have to pass the other modules to cythonize as well. It can handle a glob pattern, list of glob patterns or a list of Distutils.Extensions objects. The latter can be handy to specify cython compiler directives

from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize

extensions = [
    Extension("verifier_c", ["verifier/verifier_c.pyx"]),
    Extension("something_else", ["foobar/something_else.pyx"] compiler_directives={'embedsignature': True}),
    ]

setup(
    ext_modules=cythonize(extensions),
)

I hope this help :)