Python venv ModuleNotFoundError

Question

(Before marking me with duplicate question votes, please note that everything that I can find on this question has to do with virtualenv, not venv)

System:

• Ubuntu Server 16.04
• Python 3.6 installed, Python 3.5 native

Background (Contrived example, so might have a typo or two, but the idea is the important part)

I have a project in the form of:

Project/
├── __init__.py
└── project
    ├── packageA
    │   ├── fileA.py
    │   └── __init__.py
    └── packageB
        ├── fileB.py
        └── __init__.py

in fileb.py, I have an import statement such as

import project.packageA.fileA

I create a venv by;

cd /path/to/Project; python3.6 -m venv .venv; source .venv/bin/activate

then I run

source project/packageB/fileB.py

This will give me an error:

ModuleNotFoundError: no module named 'project'

Attempts to address:

1. Add the path to "Project" and "project" to $PATH
2. Modify $PYTHONPATH in the .venv/bin/activate script so that sys.path now points to "Project" and "project"
3. Completely read the docs here
4. Verified that os.sys.path shows Project and project paths

Question: It must be possible to do module imports using venv or it would be of zero value -- so what am I fundamentally missing in my setup? (With viritualenv, I just used 'add2virtualenv')

[Edit - Showing more detail]

# Changes to .venv/bin/activate
PYTHONPATH="/home/steve/Temp/Project:/home/steve/Temp/Project/project:$PYTHONPATH"
export PYTHONPATH

python -c "import os; print(os.sys.path)"
['', '/home/steve/Temp/Project', '/home/steve/Temp/Project/project', .....

[Edit2 - adding packageA to PYTHONPATH Works]

If I add 'path/to/packageA' to my PYTHONPATH, the import works. To use this, I would have to add each subpackage to my project -- less than ideal for large projects.

Answer 1

It's a somewhat murky situation IMHO. My solution to this is: create a setup.py for your project (beneficial anyways), and with activated venv do a "python setup.py develop".

That will add your project to a PTH-file, and thus you can import.

Example for a setup.py, taken from the interwebs:

# from http://python-packaging.readthedocs.io/en/latest/minimal.html

from setuptools import setup

setup(name='funniest',
      version='0.1',
      description='The funniest joke in the world',
      url='http://github.com/storborg/funniest',
      author='Flying Circus',
      author_email='flyingcircus@example.com',
      license='MIT',
      packages=['funniest'],
      zip_safe=False)

Answer 2

This line

​$ source project/packageB/fileB.py

fails because

1. the import path is messed up, it includes the project folder but it should not
2. project is possibly not in your PYTHONPATH

To fix it

Step 1) fix the import statement in fileB.py, replace your import with

​import packageA.fileA

Step 2) Confirm for yourself whether you added project to PYTHONPATH by checking your bash environment

​$ echo $PYTHONPATH     # does it contain `path/to/project`?

If not temporarily fix it

​$ export PYTHONPATH=path/to/project:$PYTHONPATH   # forget `/path/to/Project` you only need `path/to/Project/project`

(Note changes to $PATH are irrelevant to Python package/module searches, so that was a wasted attempt).

Then when you run your script, it will not fail:

$ source project/packageB/fileB.py      # success?!

By the way it is better to call your python scripts with python:

$ python project/packageB/fileB.py

Finally, permanently update your virtual environment by editing the activate script in your virtual environment's bin directory. Add the PYTHONPATH export above somewhere near the top.