multiprocessing.Pool with maxtasksperchild produces equal PIDs

前端 未结 2 877
温柔的废话
温柔的废话 2021-01-02 04:55

I need to run a function in a process, which is completely isolated from all other memory, several times. I would like to use multiprocessing for that (since I

2条回答
  •  余生分开走
    2021-01-02 05:48

    observe that using chunksize=1 in a Pool map will do the pool wait for a complete round of process to finish to start a new one.

    with Pool(3, maxtasksperchild=1) as p:
        p.map(do_job, args_list, chunksize=1)
    

    For example, above the pool will wait until all the first 3 process (eg 1000,1001,1002) finish to then start the new round(1003,1004,1005)

提交回复
热议问题