General decorator to wrap try except in python?

Question

I\'d interacting with a lot of deeply nested json I didn\'t write, and would like to make my python script more \'forgiving\' to invalid input. I find myself writing involve

Answer 1

It depends on what exceptions you expect.

If your only use case is get(), you could do

item['b'] = myobject.get('key2', '')

For the other cases, your decorator approach might be useful, but not in the way you do it.

I'll try to show you:

def f(func):
   def silenceit(*args, **kwargs): # takes all kinds of arguments
      try:
         return func(*args, **kwargs) # returns func's result
      except Exeption, e:
         print('Error:', e)
         return e # not the best way, maybe we'd better return None
                  # or a wrapper object containing e.
  return silenceit # on the correct level

Nevertheless, f(some_undefined_function())won't work, because

a) f() isn't yet active at the executon time and

b) it is used wrong. The right way would be to wrap the function and then call it: f(function_to_wrap)().

A "layer of lambda" would help here:

wrapped_f = f(lambda: my_function())

wraps a lambda function which in turn calls a non-existing function. Calling wrapped_f() leads to calling the wrapper which calls the lambda which tries to call my_function(). If this doesn't exist, the lambda raises an exception which is caught by the wrapper.

This works because the name my_function is not executed at the time the lambda is defined, but when it is executed. And this execution is protected and wrapped by the function f() then. So the exception occurs inside the lambda and is propagated to the wrapping function provided by the decorator, which handles it gracefully.

This move towards inside the lambda function doesn't work if you try to replace the lambda function with a wrapper like

g = lambda function: lambda *a, **k: function(*a, **k)

followed by a

f(g(my_function))(arguments)

because here the name resolution is "back at the surface": my_function cannot be resolved and this happens before g() or even f() are called. So it doesn't work.

And if you try to do something like

g(print)(x.get('fail'))

it cannot work as well if you have no x, because g() protects print, not x.

If you want to protect x here, you'll have to do

value = f(lambda: x.get('fail'))

because the wrapper provided by f() calls that lambda function which raises an exception which is then silenced.

Answer 2

in your case you first evaluate the value of the meow call (which doesn't exist) and then wrap it in the decorator. this doesn't work that way.

first the exception is raised before it was wrapped, then the wrapper is wrongly indented (silenceit should not return itself). You might want to do something like:

def hardfail():
  return meow() # meow doesn't exist

def f(func):
  def wrapper():
    try:
      func()
    except:
      print 'error'
  return wrapper

softfail =f(hardfail)

output:

>>> softfail()
error

>>> hardfail()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in hardfail
NameError: global name 'meow' is not defined

anyway in your case I don't understand why you don't use a simple method such as

def get_subkey(obj, key, subkey):
  try:
    return obj.get(key).get(subkey, '')
  except AttributeError:
    return ''

and in the code:

 item['a'] = get_subkey(myobject, 'key', 'subkey')

Edited:

In case you want something that will work at any depth. You can do something like:

def get_from_object(obj, *keys):
  try:
    value = obj
    for k in keys:
        value = value.get(k)
    return value
  except AttributeError:
    return ''

That you'd call:

>>> d = {1:{2:{3:{4:5}}}}
>>> get_from_object(d, 1, 2, 3, 4)
5
>>> get_from_object(d, 1, 2, 7)
''
>>> get_from_object(d, 1, 2, 3, 4, 5, 6, 7)
''
>>> get_from_object(d, 1, 2, 3)
{4: 5}

And using your code

item['a'] = get_from_object(obj, 2, 3) 

By the way, on a personal point of view I also like @cravoori solution using contextmanager. But this would mean having three lines of code each time:

item['a'] = ''
with ignored(AttributeError):
  item['a'] = obj.get(2).get(3) 

Answer 3

You could use a defaultdict and the context manager approach as outlined in Raymond Hettinger's PyCon 2013 presentation

from collections import defaultdict
from contextlib import contextmanager

@contextmanager
def ignored(*exceptions):
  try:
    yield
  except exceptions:
    pass 

item = defaultdict(str)

obj = dict()
with ignored(Exception):
  item['a'] = obj.get(2).get(3) 

print item['a']

obj[2] = dict()
obj[2][3] = 4

with ignored(Exception):
  item['a'] = obj.get(2).get(3) 

print item['a']

Answer 4

There are lots of good answers here, but I didn't see any that address the question of whether you can accomplish this via decorators.

The short answer is "no," at least not without structural changes to your code. Decorators operate at the function level, not on individual statements. Therefore, in order to use decorators, you would need to move each of the statements to be decorated into its own function.

But note that you can't just put the assignment itself inside the decorated function. You need to return the rhs expression (the value to be assigned) from the decorated function, then do the assignment outside.

To put this in terms of your example code, one might write code with the following pattern:

@return_on_failure('')
def computeA():
    item['a'] = myobject.get('key').METHOD_THAT_DOESNT_EXIST()

item["a"] = computeA()

return_on_failure could be something like:

def return_on_failure(value):
  def decorate(f):
    def applicator(*args, **kwargs):
      try:
         f(*args,**kwargs)
      except:
         print('Error')

    return applicator

  return decorate

Answer 5

Why not just use cycle?

for dst_key, src_key in (('a', 'key'), ('b', 'key2')):
    try:
        item[dst_key] = myobject.get(src_key).get('subkey')
    except Exception:  # or KeyError?
        item[dst_key] = ''

Or if you wish write a little helper:

def get_value(obj, key):
    try:
        return obj.get(key).get('subkey')
    except Exception:
        return ''

Also you can combine both solutions if you have a few places where you need to get value and helper function would be more reasonable.

Not sure that you actually need a decorator for your problem.

Answer 6

Since you're dealing with lots of broken code, it may be excusable to use eval in this case.

def my_eval(code):
  try:
    return eval(code)
  except:  # Can catch more specific exceptions here.
    return ''

Then wrap all your potentially broken statements:

item['a'] = my_eval("""myobject.get('key').get('subkey')""")
item['b'] = my_eval("""myobject.get('key2')""")
item['c'] = my_eval("""func1(myobject)""")