Applying two functions to two lists simultaneously using Pool and multiprocessing

Question

I have a (large) list with male and female agentes.

I want to apply different functions to each.

How can I use Pool in such a case? Given that the agents are independent of each other.

An example would be:

males = ['a', 'b', 'c']
females = ['d', 'e', 'f']
for m in males:
    func_m(m)
for f in females:
    func_f(f)

I started like that:

from multiprocessing import Pool
p = Pool(processes=2)
p.map() # Here is the problem

I would like to have something like:

p.ZIP(func_f for f in females, func_m for m in males) # pseudocode

Answer 1

This is possible to launch the computation asynchronously using map_async. This launches all the job needed and you can then collect them in a single list using the get method on the results.

from multiprocessing import Pool

pool = Pool(4)
res_male = pool.map_async(func_m, males)
res_females = pool.map_async(fun_f, females)

res = res_male.get()
res.extend(res_females.get())

You could also look to the more modern concurrent.futures API which is more intuitive for this kind of computations.

Answer 2

Not a great answer, but the first thing that came to mind:

import itertools
f = lambda t: func_m(t[0]) if t[1] else func_f(t[0])
p.map(f, itertools.chain(((0,x) for x in females), ((1,x) for x in males)))