Multiprocessing Error with Results

问题

I have a small problem in a big program, so I made a small example, which shows my problem:

import multiprocessing

class class1():
    def classfunction1(self, a):
        self.x = a

class class2():
    def classfunction2(self, a):
        self.y = a

def test(i):
    print("I'm in the Testfunction")
    b = i * class1.x * class2.y

    return b

class1 = class1()
class2 = class2()

if __name__ == "__main__":
    x = 1
    y = 2
    class1.classfunction1(x)
    class2.classfunction2(y)
    print("This variable is callable", class1.x)
    print("And this one is also callable", class2.y)
    counter = []
    for i in range(10):
        counter.append(i)
    pool = multiprocessing.Pool(processes=4)
    results = pool.imap(test, counter)
    pool.close()
    pool.join()
    #resultslist = list(results)

When I use the last line resultslist = list(results) I get the follow Error:

multiprocessing.pool.RemoteTraceback:

Traceback (most recent call last):
  File "C:\Program Files\Anaconda3\lib\multiprocessing\pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "C:\Users\example2.py", line 24, in test
    b = i * class1.x * class2.y
AttributeError: 'class1' object has no attribute 'x'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\example2.py", line 43, in <module>
    resultslist = list(results)
  File "C:\Program Files\Anaconda3\lib\multiprocessing\pool.py", line 695, in next
    raise value
AttributeError: 'class1' object has no attribute 'x'

It is necessary that the commands class1.classfunction1(x) and class2.classfunction2(y) are in the if__name__=="__main__". I need the basic structure of this script, so please don't do too many changes (if possible).

回答1:

The problem is you're trying to access the x attribute of the class1 instance of class1(!) that has never been created in the worker subprocesses because class1.classfunction1() isn't called in them.

To fix that issue I added an init() function and call it both in the main process and specify it as the initializer function for each of the subprocesses created by the Pool:

import multiprocessing

class class1():
    def classfunction1(self, a):
        self.x = a

class class2():
    def classfunction2(self, a):
        self.y = a

def test(i):
    print("I'm in the Testfunction")
    b = i * class1.x * class2.y

    return b

def init():  # added
    global class1, class2
    class1 = class1()
    class2 = class2()
    x = 1
    y = 2
    class1.classfunction1(x)
    class2.classfunction2(y)

if __name__ == "__main__":
    init()  # explicit call here
    print("This variable is callable", class1.x)
    print("And this one is also callable", class2.y)
    counter = []
    for i in range(10):
        counter.append(i)
    pool = multiprocessing.Pool(initializer=init, processes=4)  # implicit call
    results = pool.imap(test, counter)
    pool.close()
    pool.join()
    resultslist = list(results)
    print(resultslist)

回答2:

Worked on my machine python 2.7.10

  ('This variable is callable', 1)
  ('And this one is also callable', 2)
  I'm in the Testfunction
  I'm in the Testfunction
  I'm in the Testfunction
  I'm in the Testfunction
  I'm in the Testfunction
  I'm in the Testfunction
  I'm in the Testfunction
  I'm in the Testfunction
  I'm in the Testfunction
  I'm in the Testfunction

 [0, 2, 4, 6, 8, 10, 12, 14, 16, 18]

来源：https://stackoverflow.com/questions/41003242/multiprocessing-error-with-results

标签

python

function

class

multiprocessing

pool