How can type 'SwigPyObject be registered using copy_reg.pickle in Python?

為{幸葍}努か 提交于 2019-12-14 03:48:44

问题


I have a class method which I would like to use with multiprocessing.Pool for parallelisation. As class instances are not pickleable, I have used the following:

import copy_reg
import types

def _reduce_method(m):
    if m.im_self is None:
        return getattr, (m.im_class, m.im_func.func_name)
    else:
        return getattr, (m.im_self, m.im_func.func_name)

copy_reg.pickle(types.MethodType, _reduce_method)

This works no problem. However, within my class I use the GDAL module (https://pypi.org/project/GDAL/) for manipulating geospatial images and data. So now, I get the following error:

cPickle.PicklingError: Can't pickle <type 'SwigPyObject'>: attribute lookup __builtin__.SwigPyObject failed

I'm using Python 2.7.10. I know that I could solve my Pool problem using Python 3, or by using Pathos instead of Multiprocessing, but I can't do either of these easily because of network restrictions on my machine.

I've spent a bit of time searching for a solution, but to no avail. I've found some potential solutions (e.g. How to make my SWIG extension module work with Pickle?) but I'm not sure how to implement them, as I do not deliberately create a SWIG object in my code, but it must come in at some point due to GDAL.

Is there a way of registering type 'SwigPyObject' as pickleable using copy_reg, as I did with type 'instancemethod' above?

来源:https://stackoverflow.com/questions/51038892/how-can-type-swigpyobject-be-registered-using-copy-reg-pickle-in-python

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!