multiprocessing package in interactive Python

Question

I have the following code test.py:

#multiprocessing in the interactive Python 

import time
from multiprocessing import Process, Pipe

def MyProcess(a):

            


        

Answer 1

Correct, you can't use multiprocessing from the interpreter… primarily because pickle doesn't know how to serialize interactive functions. However, if you use a multiprocessing fork, called pathos.multiprocessing, you can do what you want from the interpreter. This works because pathos.multiprocessing uses dill, and dill knows how to serialize functions (and other objects) that are defined in the interpreter.

>>> from pathos.multiprocessing import ProcessingPool as Pool
>>> 
>>> p = Pool(4)
>>> def squared(x):
...   return x**2
... 
>>> def pow(x,y):
...   return x**y
... 
>>> a = range(10)
>>> b = range(10,0,-1)
>>> 
>>> p.map(squared, a)
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
>>> res = p.amap(pow, a, b)
>>> print "asynchronous, and with multiple inputs!"
asynchronous, and with multiple inputs!
>>> res.get()
[0, 1, 256, 2187, 4096, 3125, 1296, 343, 64, 9]

Get pathos here: https://github.com/uqfoundation