Plugin architecture - Plugin Manager vs inspecting from plugins import *

Question

I\'m currently writing an application which allows the user to extend it via a \'plugin\' type architecture. They can write additional python classes based on a BaseClass o

Answer 1

The approach from will-hart was the most useful one to me! For i needed more control I wrapped the Plugin Base class in a function like:

def get_plugin_base(name='Plugin',
                       cls=object,
                       metaclass=PluginMount):

    def iter_func(self):
        for mod in self._models:
            yield mod

    bases = not isinstance(cls, tuple) and (cls,) or cls

    class_dict = dict(
        _models=None,
        session=None
    )

    class_dict['__iter__'] = iter_func

    return metaclass(name, bases, class_dict)

and then:

from plugin import get_plugin_base
Plugin = get_plugin_base()

This allows to add additional baseclasses or switching to another metaclass.

Answer 2

The metaclass approach is useful for this issue in Python < 3.6 (see @quasoft's answer for Python 3.6+). It is very simple and acts automatically on any imported module. In addition, complex logic can be applied to plugin registration with very little effort. It requires:

The metaclass approach works like the following:

1) A custom PluginMount metaclass is defined which maintains a list of all plugins

2) A Plugin class is defined which sets PluginMount as its metaclass

3) When an object deriving from Plugin - for instance MyPlugin is imported, it triggers the __init__ method on the metaclass. This registers the plugin and performs any application specific logic and event subscription.

Alternatively if you put the PluginMount.__init__ logic in PluginMount.__new__ it is called whenver a new instance of a Plugin derived class is created.

class PluginMount(type):
    """
    A plugin mount point derived from:
        http://martyalchin.com/2008/jan/10/simple-plugin-framework/
    Acts as a metaclass which creates anything inheriting from Plugin
    """

    def __init__(cls, name, bases, attrs):
        """Called when a Plugin derived class is imported"""

        if not hasattr(cls, 'plugins'):
            # Called when the metaclass is first instantiated
            cls.plugins = []
        else:
            # Called when a plugin class is imported
            cls.register_plugin(cls)

    def register_plugin(cls, plugin):
        """Add the plugin to the plugin list and perform any registration logic"""

        # create a plugin instance and store it
        # optionally you could just store the plugin class and lazily instantiate
        instance = plugin()

        # save the plugin reference
        cls.plugins.append(instance)

        # apply plugin logic - in this case connect the plugin to blinker signals
        # this must be defined in the derived class
        instance.register_signals()

Then a base plugin class which looks like:

class Plugin(object):
    """A plugin which must provide a register_signals() method"""
    __metaclass__ = PluginMount

Finally, an actual plugin class would look like the following:

class MyPlugin(Plugin):
    def register_signals(self):
        print "Class created and registering signals"

    def other_plugin_stuff(self):
        print "I can do other plugin stuff"

Plugins can be accessed from any python module that has imported Plugin:

for plugin in Plugin.plugins:
    plugin.other_plugin_stuff()

See the full working example

Answer 3

Since Python 3.6 a new class method __init_subclass__ is added, that is called on a base class, whenever a new subclass is created.

This method can further simplify the solution offered by will-hart above, by removing the metaclass.

The __init_subclass__ method was introduced with PEP 487: Simpler customization of class creation. The PEP comes with a minimal example for a plugin architecture:

It is now possible to customize subclass creation without using a metaclass. The new __init_subclass__ classmethod will be called on the base class whenever a new subclass is created:

class PluginBase:
    subclasses = []

    def __init_subclass__(cls, **kwargs):
        super().__init_subclass__(**kwargs)
        cls.subclasses.append(cls)

class Plugin1(PluginBase):
    pass

class Plugin2(PluginBase):
    pass

The PEP example above stores references to the classes in the Plugin.plugins field.

If you want to store instances of the plugin classes, you can use a structure like this:

class Plugin:
    """Base class for all plugins. Singleton instances of subclasses are created automatically and stored in Plugin.plugins class field."""
    plugins = []

    def __init_subclass__(cls, **kwargs):
        super().__init_subclass__(**kwargs)
        cls.plugins.append(cls())

class MyPlugin1(Plugin):
    def __init__(self):
        print("MyPlugin1 instance created")

    def do_work(self):
        print("Do something")

class MyPlugin2(Plugin):
    def __init__(self):
        print("MyPlugin2 instance created")

    def do_work(self):
        print("Do something else")

for plugin in Plugin.plugins:
    plugin.do_work()

which outputs:

MyPlugin1 instance created
MyPlugin2 instance created
Do something
Do something else