Handle generator exceptions in its consumer

Question

This is a follow-up to Handle an exception thrown in a generator and discusses a more general problem.

I have a function that reads data in different formats. All fo

Answer 1

I like the given answer with the Frozen stuff. Based on that idea I came up with this, solving two aspects I did not yet like. The first was the patterns needed to write it down. The second was the loss of the stack trace when yielding an exception. I tried my best to solve the first by using decorators as good as possible. I tried keeping the stack trace by using sys.exc_info() instead of the exception alone.

My generator normally (i.e. without my stuff applied) would look like this:

def generator():
  def f(i):
    return float(i) / (3 - i)
  for i in range(5):
    yield f(i)

If I can transform it into using an inner function to determine the value to yield, I can apply my method:

def generator():
  def f(i):
    return float(i) / (3 - i)
  for i in range(5):
    def generate():
      return f(i)
    yield generate()

This doesn't yet change anything and calling it like this would raise an error with a proper stack trace:

for e in generator():
  print e

Now, applying my decorators, the code would look like this:

@excepterGenerator
def generator():
  def f(i):
    return float(i) / (3 - i)
  for i in range(5):
    @excepterBlock
    def generate():
      return f(i)
    yield generate()

Not much change optically. And you still can use it the way you used the version before:

for e in generator():
  print e

And you still get a proper stack trace when calling. (Just one more frame is in there now.)

But now you also can use it like this:

it = generator()
while it:
  try:
    for e in it:
      print e
  except Exception as problem:
    print 'exc', problem

This way you can handle in the consumer any exception raised in the generator without too much syntactic hassle and without losing stack traces.

The decorators are spelled out like this:

import sys

def excepterBlock(code):
  def wrapper(*args, **kwargs):
    try:
      return (code(*args, **kwargs), None)
    except Exception:
      return (None, sys.exc_info())
  return wrapper

class Excepter(object):
  def __init__(self, generator):
    self.generator = generator
    self.running = True
  def next(self):
    try:
      v, e = self.generator.next()
    except StopIteration:
      self.running = False
      raise
    if e:
      raise e[0], e[1], e[2]
    else:
      return v
  def __iter__(self):
    return self
  def __nonzero__(self):
    return self.running

def excepterGenerator(generator):
  return lambda *args, **kwargs: Excepter(generator(*args, **kwargs))