Assembling Python module on fly, dynamic import

问题

I'm trying to get myself familiar with importlib hooks. I want to implement an ability to directly import non-pythonic files written in other language, and maintain source maps, so raising SyntaxErrors with line numbers will still give meaningful stacktraces.

My approach to loading foreign files is assembling Pythonic source, then compiling it and executing it in the desired context.

I've read in the documentation that implementing importlib.abc.SourceLoader seems to be my choice — however, the only method that is being called is exec_module. At that stage, to my understanding, module object is fully ready. So why are get_source, get_data, get_code and others are not called?

My stub implementation:

import sys
import os

import importlib.abc
import importlib.machinery

class MyFinder(importlib.abc.MetaPathFinder):
    def __init__(self):
        pass

    def find_spec(self, fullname, path, target=None):
        print('find_spec', fullname, path, target)

        # filename = '{}.npy'.format(fullname)
        # if not os.path.exists(filename):
        #     return

        if fullname != 'foobar':
            return

        filename = 'foobar://ponyworld/foo.py'

        spec = importlib.machinery.ModuleSpec(
            name = fullname,
            loader = MyLoader(fullname, path, target),
            origin = filename,
            loader_state = 1234,
            is_package = False,
        )

        return spec

class MyLoader(importlib.abc.SourceLoader):
    def __init__(self, fullname, path, target):
        pass

    def get_data(self, path):
        print('get_data', path)

    def get_filename(self, fullname):
        print('get_filename', fullname)

    def path_stats(self, path):
        print('path_stats', path)

    def set_data(self, path, data):
        print('set_data', path, data)

    def get_code(self, fullname):
        print('get_code', fullname)

    def exec_module(self, module):
        print('exec_module', module)
        print(dir(module))

    def get_source(self, fullname):
        print('get_source', fullname)

    def is_package(self, fullname):
        print('is_package', fullname)

sys.meta_path.append(MyFinder())

# import fake module to see if it works
import foobar

回答1:

I misunderstood documentation and overlooked that importlib.abc.SourceLoader already provides implementations for get_code, exec_module, load_module, get_source and is_package. Those are to be overridden only in need, and only get_data and get_filename are the required minimum.

来源：https://stackoverflow.com/questions/29584902/assembling-python-module-on-fly-dynamic-import