Initializing and destroying Python multiprocessing workers

Question

I have a model which I\'m calling many times from Python. The model takes a long time to startup and shutdown, but only a short time to process the input data (which can be

Answer 1

Solution of johnthexiii would kill the model at first run of worker function. You could offer a seperate destroy function:

import time

def g(x): # destroy function
    del model
    time.sleep(1) # to ensure, this worker does not pick too many
    return None

Before pool.close() you add

pool.map_async(g, range(2), 1) # if specified to have two processes before

I don’t think, this is a very “clean” solution, but it should work.