Multiprocessing Bomb

Question

I was working the following example from Doug Hellmann tutorial on multiprocessing:

import multiprocessing

def worker():
    \"\"\"worker function\"\"\"
            


        

Answer 1

Note that the documentation mentions that you need the if statement on windows (here).

However, the documentation doesn't say that this kills your machine almost instantly, requiring a reboot. So this can be quite confusing, especially if the use of multiprocessing happens in some function deep inside the code. No matter how deeply hidden it is, you still need the if check in the main program file. This pretty much rules out using multiprocessing in any kind of library.

multiprocessing in general seems a bit rough. It might have the interface of the thread interface, but there is just no simple way around the GIL.

For more complex parallelization problems I would also look at the subprocess module or some other libraries (like mpi4py or Parallel Python).