Getting Python's unittest results in a tearDown() method

Question

Is it possible to get the results of a test (i.e. whether all assertions have passed) in a tearDown() method? I\'m running Selenium scripts, and I\'d like to do some reporti

Answer 1

Here's a solution for those of us who are uncomfortable using solutions that rely on unittest internals:

First, we create a decorator that will set a flag on the TestCase instance to determine whether or not the test case failed or passed:

import unittest
import functools

def _tag_error(func):
    """Decorates a unittest test function to add failure information to the TestCase."""

    @functools.wraps(func)
    def decorator(self, *args, **kwargs):
        """Add failure information to `self` when `func` raises an exception."""
        self.test_failed = False
        try:
            func(self, *args, **kwargs)
        except unittest.SkipTest:
            raise
        except Exception:  # pylint: disable=broad-except
            self.test_failed = True
            raise  # re-raise the error with the original traceback.

    return decorator

This decorator is actually pretty simple. It relies on the fact that unittest detects failed tests via Exceptions. As far as I'm aware, the only special exception that needs to be handled is unittest.SkipTest (which does not indicate a test failure). All other exceptions indicate test failures so we mark them as such when they bubble up to us.

We can now use this decorator directly:

class MyTest(unittest.TestCase):
    test_failed = False

    def tearDown(self):
        super(MyTest, self).tearDown()
        print(self.test_failed)

    @_tag_error
    def test_something(self):
        self.fail('Bummer')

---

It's going to get really annoying writing this decorator all the time. Is there a way we can simplify? Yes there is!^* We can write a metaclass to handle applying the decorator for us:

class _TestFailedMeta(type):
    """Metaclass to decorate test methods to append error information to the TestCase instance."""
    def __new__(cls, name, bases, dct):
        for name, prop in dct.items():
            # assume that TestLoader.testMethodPrefix hasn't been messed with -- otherwise, we're hosed.
            if name.startswith('test') and callable(prop):
                dct[name] = _tag_error(prop)

        return super(_TestFailedMeta, cls).__new__(cls, name, bases, dct)

Now we apply this to our base TestCase subclass and we're all set:

import six  # For python2.x/3.x compatibility

class BaseTestCase(six.with_metaclass(_TestFailedMeta, unittest.TestCase)):
    """Base class for all our other tests.

    We don't really need this, but it demonstrates that the
    metaclass gets applied to all subclasses too.
    """


class MyTest(BaseTestCase):

    def tearDown(self):
        super(MyTest, self).tearDown()
        print(self.test_failed)

    def test_something(self):
        self.fail('Bummer')

There are likely a number of cases that this doesn't handle properly. For example, it does not correctly detect failed subtests or expected failures. I'd be interested in other failure modes of this, so if you find a case that I'm not handling properly, let me know in the comments and I'll look into it.

---

^*If there wasn't an easier way, I wouldn't have made _tag_error a private function ;-)