python import path: packages with the same name in different folders

Question

I am developing several Python projects for several customers at the same time. A simplified version of my project folder structure looks something like this:

Answer 1

What happens if you accidentally import code from one customer/project in another and don't notice? When you deliver it will almost certainly fail. I would adopt a convention of having PYTHONPATH set up for one project at a time, and not try to have everything you've ever written be importable at once.

You can use a wrapper script per-project to set PYTHONPATH and start python, or use scripts to switch environments when you switch projects.

Of course some projects well have dependencies on other projects (those libraries you mentioned), but if you intend for the customer to be able to import several projects at once then you have to arrange for the names to not clash. You can only have this problem when you have multiple projects on the PYTHONPATH that aren't supposed to be used together.

Answer 2

This is the solution to my problem, albeit it might not be obvious at first.

In my projects, I have now introduced a convention of one namespace per customer. In every customer folder (cust1, cust2, etc.), there is an __init__.py file with this code:

import pkgutil
__path__ = pkgutil.extend_path(__path__, __name__)

All the other __init__.py files in my packages are empty (mostly because I haven't had the time yet to find out what else to do with them).

As explained here, extend_path makes sure Python is aware there is more than one sub-package within a package, physically located elsewhere and - from what I understand - the interpreter then does not stop searching after it fails to find a module under the first package path it encounters in sys.path, but searches all paths in __path__.

I can now access all code in a consistent manner criss-cross between all projects, e.g.

from cust1.proj1.pack1.mod1 import something
from cust3.proj4.pack1.mod4 import something_else
from cust3.proj1.pack4.mod4 import yet_something_else

On a downside, I had to create an even deeper project folder structure:

/path/
  to/
    projects/
      cust1/
        proj1/
          Development/
            code/
              python/
                cust1/
                  __init__.py   <--- contains code as described above
                  proj1/
                    __init__.py <--- empty
                    pack1/
                    __init__.py <--- empty
                    mod1.py

but that seems very acceptable to me, especially considering how little effort I need to make to maintain this convention. sys.path is extended by /path/to/projects/cust1/proj1/Development/code/python for this project.

On a sidenote, I noticed that of all the __init__.py files for the same customer, the one in the path that appears first in sys.path is executed, no matter from which project I import something.

Answer 3

You should be using the excellent virtualenv and virtualenvwrapper tools.