How to use a python context manager inside a generator

Question

In python, should with-statements be used inside a generator? To be clear, I am not asking about using a decorator to create a context manager from a generator function. I am as

Answer 1

from the Data model entry for object.__exit__

If an exception is supplied, and the method wishes to suppress the exception (i.e., prevent it from being propagated), it should return a true value. Otherwise, the exception will be processed normally upon exit from this method.

In your __exit__ function, you're returning True which will suppress all exceptions. If you change it to return False, the exceptions will continue to be raised as normal (with the only difference being that you guarantee that your __exit__ function gets called and you can make sure to clean up after yourself)

For example, changing the code to:

def __exit__(self, exctype, value, tb):
    print "  __exit__; excptype: '%s'; value: '%s'" % (exctype, value)
    if exctype is GeneratorExit:
        return False
    return True

allows you to do the right thing and not suppress the GeneratorExit. Now you only see the attribute error. Maybe the rule of thumb should be the same as with any Exception handling -- only intercept Exceptions if you know how to handle them. Having an __exit__ return True is on par (maybe slightly worse!) than having a bare except:

try:
   something()
except: #Uh-Oh
   pass

---

Note that when the AttributeError is raised (and not caught), I believe that causes the reference count on your generator object to drop to 0 which then triggers a GeneratorExit exception within the generator so that it can clean itself up. Using my __exit__, play around with the following two cases and hopefully you'll see what I mean:

try:
    for item in foo(10):
        print 'Fail - val: %d' % item.val
        item.not_an_attribute
except AttributeError:
    pass

print "Here"  #No reference to the generator left.  
              #Should see __exit__ before "Here"

and

g = foo(10)
try:
    for item in g:
        print 'Fail - val: %d' % item.val
        item.not_an_attribute
except AttributeError:
    pass

print "Here"
b = g  #keep a reference to prevent the reference counter from cleaning this up.
       #Now we see __exit__ *after* "Here"