Debugging Python Fatal Error: GC Object already Tracked

问题

My python code has been crashing with error 'GC Object already Tracked' . Trying to figure out the best approach to debug this crashes.

OS : Linux.

Is there a proper way to debug this issue.

There were couple of suggestions in the following article. Python memory debugging with GDB

Not sure which approach worked for the author.

Is there a way to generate memory dumps in such scenario which could be analyzed. Like in Windows world.

Found some article on this. But not entirely answers my question: http://pfigue.github.io/blog/2012/12/28/where-is-my-core-dump-archlinux/

回答1:

Found out the reason for this issue in my scenario (not necessarily the only reason for the GC object crash). I used the GDB and Core dumps to debug this issue.

I have Python and C Extension Code (in shared object). Python code registers a Callback routine with C Extension code. In a certain workflow a thread from C Extension code was calling the registered Call back routine in Python code.

This usually worked fine but when multiple threads did the same action concurrently it resulted in the Crash with 'GC Object already tracked'.

Synchronizing the access to python objects for multiple thread does resolve this issue.

Thanks to any responded to this.

回答2:

I ran into this problem using boost::python when our C++ code would trigger a python callback. I would occasionally get "GC object already tracked" and the program would terminate.

I was able to attach GDB to the process prior to triggering the error. One interesting thing, in the python code we were wrapping the callback with a functools partial, which was actually masking where the real error was occuring. After replacing the partial with a simple callable wrapper class. The "GC object already tracked error" no longer popped up, instead I was now just getting a segfault.

In our boost::python wrapper, we had lambda functions to handle a C++ callback and the lambda function captured the boost::python::object callback function. It turned out, for whatever reason, in the destructor for the lambda, it wasn't always properly acquiring the GIL when destroying the boost::python::object which was causing the segfault.

The fix was to not use a lambda function, but instead create a functor that makes sure to acquire the GIL in the destructor prior to calling PyDECREF() on the boost::python::object.

class callback_wrapper
{
public:
    callback_wrapper(object cb): _cb(cb), _destroyed(false) {
    }

    callback_wrapper(const callback_wrapper& other) {
        _destroyed = other._destroyed;
        Py_INCREF(other._cb.ptr());
        _cb = other._cb;
    }

    ~callback_wrapper() {
        std::lock_guard<std::recursive_mutex> guard(_mutex);
        PyGILState_STATE state = PyGILState_Ensure();
        Py_DECREF(_cb.ptr());
        PyGILState_Release(state);
        _destroyed = true;
    }

    void operator ()(topic_ptr topic) {
        std::lock_guard<std::recursive_mutex> guard(_mutex);
        if(_destroyed) {
            return;
        }
        PyGILState_STATE state = PyGILState_Ensure();
        try {
            _cb(topic);
        }
        catch(error_already_set) { PyErr_Print(); }
        PyGILState_Release(state);
    }

    object _cb;
    std::recursive_mutex _mutex;
    bool _destroyed;
};

回答3:

The problem is that you try to add an object to Python's cyclic garbage collector tracking twice.

Check out this bug, specifically:

The documentation for supporting cyclic garbage collection in Python
My documentation patch for the issue and
My explanation in the bug report itself

Long story short: if you set Py_TPFLAGS_HAVE_GC and you are using Python's built-in memory allocation (standard tp_alloc/tp_free), you don't ever have to manually call PyObject_GC_Track() or PyObject_GC_UnTrack(). Python handles it all behind your back.

Unfortunately, this is not documented very well, at the moment. Once you've fixed the issue, feel free to chime in on the bug report (linked above) about better documentation of this behavior.

来源：https://stackoverflow.com/questions/23178606/debugging-python-fatal-error-gc-object-already-tracked

标签

python

crash

garbage-collection