Import python modules in the background in REPL

Question

Some python modules, notably matplotlib, take a long time to load

start = datetime.datetime.now(); import numpy, pandas, matplotlib, sklearn; datetime.datetime.now() - start

takes half a second with cached files, and several seconds for non-cached files. Is there a way to load these modules in the background, when in the Python interpreter?

Answer 1

You can import modules in separate threads. Here is the solution.

Create a file load_modules.py:

from concurrent.futures import ThreadPoolExecutor
import importlib
import sys

modules_to_load = ['numpy', 'pandas', 'matplotlib']


def do_import(module_name):
    thismodule = sys.modules[__name__]

    module = importlib.import_module(module_name)
    setattr(thismodule, module_name, module)
    print(module_name, 'imported')


executor = ThreadPoolExecutor()
for module_name in modules_to_load:
    executor.submit(do_import, module_name)

Then you can start interpreter with a command:

python -ic "exec(open(\"load_modules.py\").read(), globals())"

Or just run

exec(open("load_modules.py").read(), globals())

in your interpreter to load modules.