Python: Using `copyreg` to define reducers for types that already have reducers

Question

(Keep in mind I'm working in Python 3, so a solution needs to work in Python 3.)

I would like to use the copyreg module to teach Python how to pickle functions. When I tried to do it, the _Pickler object would still try to pickle functions using the save_global function. (Which doesn't work for unbound methods, and that's the motivation for doing this.)

It seems like _Pickler first tries to look in its own dispatch for the type of the object that you want to pickle before looking in copyreg.dispatch_table. I'm not sure if this is intentional.

Is there any way for me to tell Python to pickle functions with the reducer that I provide?

Answer 1

The following hack seems to work in Python 3.1...:

import copyreg
def functionpickler(f):
  print('pickling', f.__name__)
  return f.__name__

ft = type(functionpickler)
copyreg.pickle(ft, functionpickler)

import pickle
pickle.Pickler = pickle._Pickler
del pickle.Pickler.dispatch[ft]

s = pickle.dumps(functionpickler)
print('Result is', s)

Out of this, the two hackish lines are:

pickle.Pickler = pickle._Pickler
del pickle.Pickler.dispatch[ft]

You need to remove the dispatch entry for functions' type because otherwise it preempts the copyreg registration; and I don't think you can do that on the C-coded Pickler so you need to set it to the Python-coded one.

It would be a bit less of a hack to subclass _Pickler with a class of your own which makes its own dispatch (copying the parent's and removing the entry for the function type), and then use your subclass specifically (and its dump method) rather than pickle.dump; however it would also be a bit less convenient that this monkeypatching of pickle itself.