Is pytest supposed to collect tests from dependency modules in a virtual environtment?

问题

I am attempting to setup a project on another laptop than my typical development machine. This project has several pytest-based tests that I have written over the lifetime of the project. When I run

$ pytest -k tests/my_test.py

I get a list of errors from sqlalchemy tests like the following:

_ ERROR collecting env/lib64/python3.5/site-packages/sqlalchemy/testing/suite/test_update_delete.py _
env/lib/python3.5/site-packages/py/_path/local.py:662: in pyimport
    __import__(modname)
env/lib/python3.5/site-packages/sqlalchemy/testing/suite/__init__.py:2: in <module>
    from sqlalchemy.testing.suite.test_cte import *
<frozen importlib._bootstrap>:968: in _find_and_load
    ???
<frozen importlib._bootstrap>:957: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:664: in _load_unlocked
    ???
<frozen importlib._bootstrap>:634: in _load_backward_compatible
    ???
env/lib/python3.5/site-packages/_pytest/assertion/rewrite.py:211: in load_module
    py.builtin.exec_(co, mod.__dict__)
env/lib/python3.5/site-packages/sqlalchemy/testing/suite/test_cte.py:11: in <module>
    class CTETest(fixtures.TablesTest):
env/lib/python3.5/site-packages/sqlalchemy/testing/suite/test_cte.py:99: in CTETest
    @testing.requires.ctes_with_update_delete
E   AttributeError: 'NoneType' object has no attribute 'ctes_with_update_delete'

Why is pytest collecting tests from a dependency? Is it supposed to do this? If not, how can I fix it?

The laptop where I am setting up this project is running Ubuntu 18.04. I created a virtualenv with Python 3.5.6 and ran pip install -r requirements in the virtualenv without any errors, including sqlalchemy and pymssql.

回答1:

I believe the problem is how you're invoking it. pytest -k tests/my_test.py probably isn't what you want to do: pytest tests/my_test.py probably is, though, as this will invoke the lone test file tests/my_test.py (no recursion will happen: including into your environment site packages). The -k flag is for expressions (so you could run all tests with "http" in the name, for instance).

It doesn't make a lot of sense, but somehow the runner is grabbing some sqlalchemy tests (or they're failing on import, I'm not really sure). Doesn't seem like they should match (even though the runner would recurse into your env directory), but I think it's a bit besides the point. You should be able to fix your issue by simply invoking pytest without the -k option.

P.S. I can't reproduce the behavior you're seeing on my windows box.

HTH.

回答2:

To answer the question in the title, no.

This may be a symptom of creating your venv and installing packages in a directory that is not ignored. Keep them somewhere ignored, like ~/.venvs/ or whatever.

Alternatively, specifically --ignore=env to exclude the site, or otherwise customize your test discovery in config file:

# content of pytest.ini
[pytest]
norecursedirs = env

回答3:

When pytest is invoked, it scans all the child dirs in the project root dir, looking for tests; when you place the project-specific virtual env into project root dir, it will be also scanned without making an exception. This can lead to unwanted tests being included in the test run (for example, when you have dependencies like numpy or pandas that include tests in the distribution).

To circumvent this, pytest offers two configuration options:

norecursedirs - holds directories that will be excluded from scanning. Use this option when you are looking for the pattern "include all, exclude selected". See also wim's answer for the explanation and usage example. By default, norecursedirs is set to .*', 'build', 'dist', 'CVS', '_darcs', '{arch}', '*.egg', so beware that when you override this option, the defaults are gone and you have to add them back.
testpaths - holds directories that should only be considered for the scan, so this is basically the opposite to what norecursedirs is doing. Use this option when looking for the pattern "exclude all, include selected". This option also adds some minor or more significant speedup to the test discovery, depending on what you keep in the project root - most of the sudbirectories won't be traversed at all and the tests run starts sooner.

Usage: either place the options in the pytest.ini/setup.cfg/tox.ini:
```
[tool:pytest]
testpaths = tests othertests doc
```
or pass them via --override-ini from command line.
```
pytest -o "testpaths=tests othertests doc" ...
```

Moving the env dir away from the project sources is a matter of choice and depends on what dev environment you have. direnv, pipenv etc. may have different views on where the envs should reside, but personally I don't see any problems in keeping the envs in the project root.

来源：https://stackoverflow.com/questions/52937050/is-pytest-supposed-to-collect-tests-from-dependency-modules-in-a-virtual-environ

标签

python

sqlalchemy

pytest