How to save traceback / sys.exc_info() values in a variable?

Question

I want to save the name of the error and the traceback details into a variable. Here\'s is my attempt.

import sys
try:
    try:
        print x
    except E         


        

Answer 1

Be careful when you take the exception object or the traceback object out of the exception handler, since this causes circular references and gc.collect() will fail to collect. This appears to be of a particular problem in the ipython/jupyter notebook environment where the traceback object doesn't get cleared at the right time and even an explicit call to gc.collect() in finally section does nothing. And that's a huge problem if you have some huge objects that don't get their memory reclaimed because of that (e.g. CUDA out of memory exceptions that w/o this solution require a complete kernel restart to recover).

In general if you want to save the traceback object, you need to clear it from references to locals(), like so:

import sys, traceback, gc
type, val, tb = None, None, None
try:
    myfunc()
except:
    type, val, tb = sys.exc_info()
    traceback.clear_frames(tb)
# some cleanup code
gc.collect()
# and then use the tb:
if tb:
    raise type(val).with_traceback(tb)

In the case of jupyter notebook, you have to do that at the very least inside the exception handler:

try:
    myfunc()
except:
    type, val, tb = sys.exc_info()
    traceback.clear_frames(tb)
    raise type(val).with_traceback(tb)
finally:
    # cleanup code in here
    gc.collect()

Tested with python 3.7.

p.s. the problem with ipython or jupyter notebook env is that it has %tb magic which saves the traceback and makes it available at any point later. And as a result any locals() in all frames participating in the traceback will not be freed until the notebook exits or another exception will overwrite the previously stored backtrace. This is very problematic. It should not store the traceback w/o cleaning its frames. Fix submitted here.