How to pass a function with more than one argument to python concurrent.futures.ProcessPoolExecutor.map()?

前端 未结 3 651
执笔经年
执笔经年 2021-01-04 02:16

I would like concurrent.futures.ProcessPoolExecutor.map() to call a function consisting of 2 or more arguments. In the example below, I have resorted to using a

3条回答
  •  既然无缘
    2021-01-04 02:45

    (1) No need to make a list. You can use itertools.repeat to create an iterator that just repeats the some value.

    (2) You need to pass a named function to map because it will be passed to the subprocess for execution. map uses the pickle protocol to send things, lambdas can't be pickled and therefore they can't be part of the map. But its totally unnecessary. All your lambda did was call a 2 parameter function with 2 parameters. Remove it completely.

    The working code is

    import concurrent.futures as cf
    import itertools
    
    nmax = 10
    numberlist = range(nmax)
    workers = 3
    
    def _findmatch(listnumber, ref):    
        print('def _findmatch(listnumber, ref):')
        x=''
        listnumber=str(listnumber)
        ref = str(ref)
        print('listnumber = {0} and ref = {1}'.format(listnumber, ref))
        if ref in listnumber:
            x = listnumber
        print('x = {0}'.format(x))
        return x 
    
    with cf.ProcessPoolExecutor(max_workers=workers) as executor:
        #for n in executor.map(_findmatch, numberlist):
        for n in executor.map(_findmatch, numberlist, itertools.repeat(5)):
            print(type(n))
            print(n)
            #if str(ref[0]) in n:
            #    print('match')
    

提交回复
热议问题