Finding exception in python multiprocessing

问题

I have a bit of python code that looks like this:

procs = cpu_count()-1
if serial or procs == 1:
    results = map(do_experiment, experiments)
else:
    pool = Pool(processes=procs)     
    results = pool.map(do_experiment, experiments)

It runs fine when I set the serial flag, but it gives the following error when the Pool is used. When I try to print something from do_experiment nothing shows up, so I can't try/catch there and print a stack trace.

Exception in thread Thread-2:
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 530, in __bootstrap_inner
    self.run()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 483, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/pool.py", line 285, in _handle_tasks
    put(task)
TypeError: 'NoneType' object is not callable

What is a good way to proceed debugging this?

回答1:

I went back in my git history until I found a commit where things were still working.

I added a class to my code that extends dict so that keys can be accessed with a . (so dict.foo in stead of dict["foo"]. Multiprocessing did not take kindly to this, using an ordinary dict solved the problem.

来源：https://stackoverflow.com/questions/10819447/finding-exception-in-python-multiprocessing