Can I define optional packages in setuptools?

Question

Currently one of my packages requires a JSON parser/encoder, and is designed to use simplejson if available falling back to the json module (in the standard library) if necessary (as benchmarks show simplejson is faster).

However, recently it's been hit & miss as to whether simplejson will install when using zc.buildout - something with the move to github, I believe. Which got me wondering; is it possible to define optional packages in my setup.py file which, if unavailable, won't stop the installation of my package?

Answer 1

optional packages at installation time.

I am assuming you are talking about your setup.py script. You could change it to have:

# mypackage/setup.py

extras = {
   'with_simplejson': ['simplejson>=3.5.3']
}

setup(
    ...
    extras_require=extras,
    ...)

then you can do either of:

• pip install mypackage,
• pip install mypackage[with_simplejson]

with the latter installing simplejson>=3.5.3.

Instead of trying to install everything and fallback to a known good version, you would want to install the subset of packages you know work.

optional packages at execution time.

Once you have two different sets of packages that could be installed, you need to make sure you can use them if they are available. E.g. for your json import:

try:
    # helpful comment saying this should be faster.
    import simplejson as json
except ImportError:
    import json

Another more complex example:

try:
    # xml is dangerous
    from defusedxml.cElementTree import parse
except ImportError:
    try:
        # cElementTree is not available in older python
        from xml.cElementTree import parse
    except ImportError:
        from xml.ElementTree import parse

But you can also find this pattern in some packages:

try:
    optional_package = None
    import optional.package as optional_package
except ImportError:
    pass

...

if optional_package:
    # do addtional behavior

Answer 2

AFAIK there is no way to define an optional package and there would be no use to do so. What do you expect when you define an optional package? That it is installed when it is not yet available? (that would somehow make it mandatory)

No, IMHO the correct way to address this is in your imports where you want to use the package. E.g:

try:
    from somespecialpackage import someapi as myapi
except ImportError:
    from basepackage import theapi as myapi

This of course requires that the two APIs are compatible, but this is the case with simplejson and the standard library json package.