setuptools and pip: choice of minimal and complete install

Question

We\'ve made a library which depends on other libraries. But there are necessary (e.g. for server batch processing) and optional dependencies (e.g. for clients with GUI).

Answer 1

So pip is actually quite picky about installing libraries with extra requirements

pip install -e ".[extra,requirements]"    # works with file paths
pip install "package[extra,requirements]" # works when downloading packages
pip install ".[extra,requirments]"        # DOES NOT WORK

I think this is down to how the RequirementsSpec parser works, and pip does some extra magic with the -e flag. Anyhow after much head banging, here's a mildly ugly workaround

pip install "file:///path/to/your/python_code#egg=SomeName[extra,requirements]"

The egg=SomeName part is basically ignored, but pip correctly picks up the extra requirements

Caveats

• Tested with pip 1.5.6 so make sure you're using a current version of pip.
• As far as I can tell, the file:/// syntax is undocumented in pip, so I'm not sure if it'll change in the future. It looks a bit like the VCS Support syntax but I was a bit surprised it worked.
• You could also get around this by running your own pypi server, but that's a bit out of scope.

Answer 2

You can install the packages in extras_require by appending the name of the recommended dependency in square brackets (i.e. [mpl] or [bn] in your case) to the package name in pip.

So to install 'mylib' with the additional requirements, you would call pip like this:

pip install 'mylib[mpl]'
pip install 'mylib[bn]'

This will first download and install the extra dependencies, and then mylib's core dependencies.

This is anologous to how you declare those dependencies with setuptools: http://pythonhosted.org/setuptools/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies (see the install_requires value in the third example)