Choose adapter dynamically depending on librarie(s) installed

Question

I am designing a library that has adapters that supports a wide-range of libraries. I want the library to dynamically choose which ever adapter that has the library it uses

Answer 1

A flexible solution, using importlib. This is a complete, working solution that i've tested.

First, the header:

import importlib
parent = 'servicelib.simple'
modules = {'.synchronous':['.alternative', '.alternative_2']}
success = False #an indicator, default is False,
#changed to True when the import succeeds.

We import the required module, set our indicator, and specify our modules. modules is a dictionary, with the key set as the default module, and the value as a list of alternatives.

Next, the import-ant part:

#Obtain the module
for default, alternatives in modules.items():
    try: #we will try to import the default module first
        mod = importlib.import_module(parent+default)
        success = True
    except ImportError: #the default module fails, try the alternatives
        for alt in alternatives:
            try: #try the first alternative, if it still fails, try the next one.
                mod = importlib.import_module(parent+alt)
                success = True
                #Stop searching for alternatives!
                break 
            except ImportError:
                    continue

print 'Success: ', success

And to have the classes, simply do:

Publisher = mod.Publisher
Consumer = mod.Consumer

With this solution, you can have multiple alternatives at once. For example, you can use both rabbitpy and pyAMPQ as your alternatives.

Note: Works with both Python 2 and Python 3.

If you have more questions, feel free to comment and ask!

Answer 2

I know two method, one is wildly used and another is my guesswork. You can choose one for your situation.

The first one, which is widely used, such as from tornado.concurrent import Future.

try:
    from concurrent import futures
except ImportError:
    futures = None

#define _DummyFuture balabala...

if futures is None:
    Future = _DummyFuture
else:
    Future = futures.Future

Then you can use from tornado.concurrent import Future in other files.

The second one, which is my guesswork, and I write simple demo, but I haven't use it in production environment because I don't need it.

import sys
try:
    import servicelibrary.simple.synchronous
except ImportError:
    import servicelibrary.simple.alternative
    sys.modules['servicelibrary.simple.synchronous'] = servicelibrary.simple.alternative

You can run the script before other script import servicelibrary.simple.synchronous. Then you can use the script as before:

from servicelibrary.simple.synchronous import Publisher
from servicelibrary.simple.synchronous import Consumer

The only thing I wonder is that what are the consequences of my guesswork.

Answer 3

You've got the right idea. Your case works because each subobject has the same sort of classes e.g. both APIs have a class called Publisher and you can just make sure the correct version is imported.

If this isn't true (if possible implementation A and B are not similar) you write your own facade, which is just your own simple API that then calls the real API with the correct methods/parameters for that library.

Obviously switching between choices may require some overhead (i don't know your case, but for instance, let's say you had two libraries to walk through an open file, and the library handles opening the file. You can't just switch to the second library in the middle of the file and expect it to start where the first library stopped). But it's just a matter of saving it:

accessmethods = {}
try:
    from modA.modB import classX as apiA_classX
    from modA.modB import classY as apiA_classY
    accessmethods['apiA'] = [apiA_classX, apiA_classY]
    classX = apiA_classX
    classY = apiA_classY
except:
    pass

try:
    from modC.modD import classX as apiB_classX
    from modC.modD import classY as apiB_classY
    accessmethods['apiB'] = [apiB_classX, apiB_classY]
    classX = apiB_classX
    classY = apiB_classY
except:
    pass

def switchMethod(method):
    global classX
    global classY
    try: 
        classX, classY = accessmethods[method]
    except KeyError as e:
        raise ValueError, 'Method %s not currently available'%method

etc.

Answer 4

The importlib.import_module might do what you need:

INSTALLED = ['syncronous', 'alternative']  

for mod_name in INSTALLED:
    try: 
        module = importlib.import_module('servicelibrary.simple.' + mod_name)
        Publisher = getattr(module, 'Publisher')

        if Publisher:
            break  # found, what we needed

    except ImportError:
        continue

I guess, this is not the most advance technique, but the idea should be clear. And you can take a look at the imp module as well.

Answer 5

Based on the answers I ended up with the following implementation for Python 2.7.

Examples are simplified for stackoverflow..

from importlib import import_module

PARENT = 'myservicelib.rabbitmq'
MODULES = ['test_adapter',
           'test_two_adapter']
SUCCESS = False

for _module in MODULES:
    try:
        __module = import_module('{0}.{1}'.format(PARENT, _module))
        Consumer = getattr(__module, 'Consumer')
        Publisher = getattr(__module, 'Publisher')
        SUCCESS = True
        break
    except ImportError:
        pass

if not SUCCESS:
    raise NotImplementedError('no supported rabbitmq library installed.')

Although, as I also had some of my projects running Python 2.6 I had to either modify the code, or include importlib. The problem with a production platform is that it isn't always easy to include new dependencies.

This is the compromise I came up with, based on __import__ instead of importlib.

It might be worth checking if sys.modules actually contains the namespace so you don't get a KeyError raised, but it is unlikely.

import sys

PARENT = 'myservicelib.rabbitmq'
MODULES = ['test_adapter',
           'test_two_adapter']
SUCCESS = False

for _module in MODULES:
    try:
        __module_namespace = '{0}.{1}'.format(PARENT, _module)
        __import__(__module_namespace)
        __module = sys.modules[__module_namespace]
        Consumer = getattr(__module, 'Consumer')
        Publisher = getattr(__module, 'Publisher')
        SUCCESS = True
        break
    except ImportError:
        pass

if not SUCCESS:
    raise NotImplementedError('no supported rabbitmq library installed.')