Python: Getting a traceback from a multiprocessing.Process

Question

I am trying to get hold of a traceback object from a multiprocessing.Process. Unfortunately passing the exception info through a pipe does not work because traceback objects

Answer 1

Since multiprocessing does print the string contents of exceptions raised in child processes, you can wrap all your child process code in a try-except that catches any exceptions, formats the relavent stack traces, and raises a new Exception that holds all the relevant information in its string:

An example of a function I use with multiprocessing.map:

def run_functor(functor):
    """
    Given a no-argument functor, run it and return its result. We can 
    use this with multiprocessing.map and map it over a list of job 
    functors to do them.

    Handles getting more than multiprocessing's pitiful exception output
    """

    try:
        # This is where you do your actual work
        return functor()
    except:
        # Put all exception text into an exception and raise that
        raise Exception("".join(traceback.format_exception(*sys.exc_info())))

What you get is a stack trace with another formatted stack trace as the error message, which helps with debugging.