Is there a simple process-based parallel map for python?

前端 未结 4 1251
小蘑菇
小蘑菇 2020-11-29 18:50

I\'m looking for a simple process-based parallel map for python, that is, a function

parmap(function,[data])

that would run function on eac

4条回答
  •  孤城傲影
    2020-11-29 19:25

    I seems like what you need is the map method in multiprocessing.Pool():

    map(func, iterable[, chunksize])

    A parallel equivalent of the map() built-in function (it supports only
    one iterable argument though). It blocks till the result is ready.
    
    This method chops the iterable into a number of chunks which it submits to the 
    process pool as separate tasks. The (approximate) size of these chunks can be 
    specified by setting chunksize to a positive integ
    

    For example, if you wanted to map this function:

    def f(x):
        return x**2
    

    to range(10), you could do it using the built-in map() function:

    map(f, range(10))
    

    or using a multiprocessing.Pool() object's method map():

    import multiprocessing
    pool = multiprocessing.Pool()
    print pool.map(f, range(10))
    

提交回复
热议问题