Distinguish between ImportError because of not found module or faulty import in module itself in python?

Question

I have a few modules in python, which are imported dynamicly and all have the same structur (plugin.py, models.py, tests.py, ...). In the managing code i want to import those submodules, but for example models.py or tests.py is not mandatory. (So i could have plugin_a.plugin and plugin_a.tests but only plugin_b.plugin).

I can check if the submodule exists by

try:
    __import__(module_name + ".tests")
except ImportError:
    pass

That will fail, if module_name+".tests" is not found, but it will also fail if the tests-module itself will try to import something, which is not found, for example because of a typo. Is there any way to check if the module exists, without importing it or make sure, the ImportError is only raised by one specific import-action?

Answer 1

You know what the import error message will look like if the module doesn't exist so just check for that:

try:
    module = module_name + '.tests'
    __import__(module)
except ImportError, e:
    if e.args and e.args[0] == 'No module named ' + module:
        print(module, 'does not exist')
    else:
        print(module, 'failed to import')

Answer 2

You can see from the length of the traceback how many levels deep the import failed. A missing .test module has a traceback with just one frame, a direct dependency failing has two frames, etc.

Python 2 version, using sys.exc_info() to access the traceback:

import sys


try:
    __import__(module_name + ".tests")
except ImportError:
    if sys.exc_info()[-1].tb_next is not None:
        print "Dependency import failed"
    else:
        print "No module {}.tests".format(module_name)

Python 3 version, where exceptions have a __traceback__ attribute:

try:
    __import__(module_name + ".tests")
except ImportError as exc:
    if exc.__traceback__.tb_next is not None:
        print("Dependency import failed")
    else:
        print("No module {}.tests".format(module_name))

Demo:

>>> import sys
>>> def direct_import_failure(name):
...     try:
...         __import__(name)
...     except ImportError:
...         return sys.exc_info()[-1].tb_next is None
... 
>>> with open('foo.py', 'w') as foo:
...     foo.write("""\
... import bar
... """)
... 
>>> direct_import_failure('bar')
True
>>> direct_import_failure('foo')
False

Answer 3

Following on from: How to check if a python module exists without importing it

The imp.find_module function will either return a 3-element tuple (file, pathname, description) or raise an ImportError.

It will not raise an error if there is a problem with the module, only if it doesn't exist.

The python docs suggest that you should first find and import the package and then use its path in second find_module, doing this recursively if necessary.

This seems a little messy to me.

The function below will check for the existence of a given module, at any level of relative import (module.py, package.module.py, package.subpackage.module.py etc.).

imp.find_module returns an open file, which you could use in imp.load_module, but this, too seems a little clunky, so I close the file so that it can be imported outside of the function.

Note that this isn't perfect. if you are looking for package.subpackage.module but actually package.subpackage is a valid module it will also return true.

import imp
import importlib

def module_exists(modulename):
    modlist = modulename.split('.')
    pathlist = None
    for mod in modlist:
        print mod
        try:
            openfile, pathname, desc = imp.find_module(mod,pathlist)
            pathlist = [pathname]
        except ImportError:
            print "Module '{}' does not exist".format(mod)
            return(False)
        else:
            print 'found {}'.format(openfile)
            if openfile:
                openfile.close()
                return(True)

if __name__ == '__main__':
    mymodule = 'parrot.type.norwegian_blue'
    if module_exists(mymodule):
        importlib.import_module(mymodule)

Note also that I'm using importlib.import_module instead of __import__.

Finally, note that importlib is Python 2.7 and upwards

Answer 4

Is there any way to check if the module exists, without importing it or make sure, the ImportError is only raised by one specific import-action?

There could be multiple reasons why ImportError fails because importing will evaluate the module; if there is a syntax error the module will fail to load.

To check if a module exists without loading it, use pkgutil.find_loader, like this:

>>> pkgutil.find_loader('requests')
<pkgutil.ImpLoader instance at 0x9a9ce8c>
>>> pkgutil.find_loader('foo')

It will return either a ImpLoader instance, or None if the package is not importable. You can get further details from the ImpLoader instance, like the path of the module:

>>> pkgutil.find_loader('django').filename
'/usr/local/lib/python2.7/dist-packages/django'