python multiprocessing apply_async only uses one process

前端 未结 2 1240
没有蜡笔的小新
没有蜡笔的小新 2020-12-23 19:06

I have a script that includes opening a file from a list and then doing something to the text within that file. I\'m using python multiprocessing and Pool to try to parallel

2条回答
  •  忘掉有多难
    2020-12-23 19:50

    Maybe in this case you should use map_async:

    import os
    from multiprocessing import Pool
    
    results = []
    def testFunc(file):
        message =  ("Working in Process #%d" % (os.getpid()))
        #This is just an illustration of some logic. This is not what I'm actually doing.
        for line in file:
            if 'dog' in line:
                results.append(line)
        return message
    
    if __name__=="__main__":
        print("saddsf")
        p = Pool(processes=2)
        files = ['/path/to/file1.txt', '/path/to/file2.txt']
        results = p.map_async(testFunc, files)
        print(results.get())
    

提交回复
热议问题