Running pytest with cython - how to compile cython modules in pytest?

Question

I have a project organized as follows:

project
├── project
│   ├── module1
│   │   ├── api.py
│   │   ├── _cpython_foo.py
│   │   └── _cython_foo.pyx
│   └── module2
├── setup.py
└── tests
    └── module1
        ├── test_cython_foo.py
        └── test_cpython_foo.py

where api.py imports cythonized extensions:

 """api.py""""
 from _cython_foo import cython_fun

My setup script builds the .pyx source inplace correctly and I'm able to use cython_fun in the installed package:

import project.module1.api as module1
module1.cython_fun()    # OK

However, pytest complains that it cannot import the cython module since the compiled binaries are not in place until I invoke setup.

project/module1/api.py:2: in <module>
    from _cython_foo import cython_fun
E   ImportError: No module named _cython_foo

It seems like bad style to leave a bunch of precompiled binaries inside my project directory that pytest depends on, so is there a conventional way to have pytest temporarily build the cython modules for the purpose of my unit tests only?

Answer 1

I know about two ways of marrying pytest with cythonized extensions:

1. Do not invoke the tests via pytest directly; instead, use in-place test command with the setup script. To do that, install pytest-runner plugin that adds a pytest command:

   $ pip install pytest-runner
   

   Now you should be able to invoke the tests with pytest by issuing

   $ python setup.py pytest
   

   This approach, however, will require some changes in the way the command line args are passed: the pendant to command

   $ pytest --arg1 --arg2
   

   will be

   $ python setup.py pytest --addopts "--arg1 --arg2"
   

2. If you want to continue using the pytest command: install the project in development mode (ideally, in a virtualenv).

   (myenv) $ pip install --editable dir/
   

   where dir/ is the directory containing the setup.py script. Now the extensions will be prebuilt and pytest will be able to resolve them.