How to change the serialization method used by the multiprocessing module?

Question

How can I change the serialization method used by the Python multiprocessing library? In particular, the default serialization method uses the pickle

Answer 1

I believe the patch you're referring to works if you're using a multiprocessing "context" object.

Using your pickle2reducer.py, your client should start with:

import pickle2reducer
import multiprocessing as mp

ctx = mp.get_context()
ctx.reducer = pickle2reducer.Pickle2Reducer()

And ctx has the same API as multiprocessing.

Hope that helps!