how do I use key word arguments with python multiprocessing pool apply_async

Question

I'm trying to get to grips with pythons multiprocessing module, specifically the apply_async method of Pool. I'm trying to call a function with arguments and keyword arguments. If I call the function without kwargs it's fine but when I try to add in a keyword argument I get: TypeError: apply_async() got an unexpected keyword argument 'arg2' Below is the test code that I'm running

#!/usr/bin/env python
import multiprocessing
from time import sleep
def test(arg1, arg2=1, arg3=2):
    sleep(5)

if __name__ == '__main__':
    pool = multiprocessing.Pool()
    for t in range(1000):
        pool.apply_async(test, t, arg2=5)
    pool.close()
    pool.join()

How can I call the function so that it accepts keyword arguments?

Answer 1

Pass the keyword args in a dictionary (and the positional arguments in a tuple):

pool.apply_async(test, (t,), dict(arg2=5))

Answer 2

Janne's answer didn't work for me in python 2.7.11 (not sure why). The function test() was receiving the key (arg2), not the value (5).

I fixed this by creating a wrapper around test:

def test2(argsDict):
    test(**argsDict)

Then calling

pool.apply_async(test2, (t,), [dict(arg2=5)])