How does the callback function work in multiprocessing map_async?

Question

It cost me a whole night to debug my code, and I finally found this tricky problem. Please take a look at the code below.

from multiprocessing import Pool

d         


        

Answer 1

Callback is called once with the result ([[0], [0, 1]]) if you use map_async.

>>> from multiprocessing import Pool
>>> def myfunc(x):
...     return [i for i in range(x)]
... 
>>> A = []
>>> def mycallback(x):
...     print('mycallback is called with {}'.format(x))
...     A.extend(x)
... 
>>> pool=Pool()
>>> r = pool.map_async(myfunc, (1,2), callback=mycallback)
>>> r.wait()
mycallback is called with [[0], [0, 1]]
>>> print(A)
[[0], [0, 1]]

Use apply_async if you want callback to be called for each time.

pool=Pool()
results = []
for x in (1,2):
    r = pool.apply_async(myfunc, (x,), callback=mycallback)
    results.append(r)
for r in results:
    r.wait()