Getting Python's unittest results in a tearDown() method

Question

Is it possible to get the results of a test (i.e. whether all assertions have passed) in a tearDown() method? I\'m running Selenium scripts, and I\'d like to do some reporti

Answer 1

This solution is for Python versions 2.7 to 3.7 (the highest current version), without any decorators or other modification in any code before tearDown. Everything works according to the builtin classification of results. Also skipped tests or expectedFailure are recognized correctly. It evaluates the result of the current test, not a summary of all tests passed so far. Compatible also with pytest.

import unittest

class MyTest(unittest.TestCase):
    def tearDown(self):
        if hasattr(self, '_outcome'):  # Python 3.4+
            result = self.defaultTestResult()  # these 2 methods have no side effects
            self._feedErrorsToResult(result, self._outcome.errors)
        else:  # Python 3.2 - 3.3 or 3.0 - 3.1 and 2.7
            result = getattr(self, '_outcomeForDoCleanups', self._resultForDoCleanups)
        error = self.list2reason(result.errors)
        failure = self.list2reason(result.failures)
        ok = not error and not failure

        # demo:   report short info immediately (not important)
        if not ok:
            typ, text = ('ERROR', error) if error else ('FAIL', failure)
            msg = [x for x in text.split('\n')[1:] if not x.startswith(' ')][0]
            print("\n%s: %s\n     %s" % (typ, self.id(), msg))

    def list2reason(self, exc_list):
        if exc_list and exc_list[-1][0] is self:
            return exc_list[-1][1]

    # DEMO tests
    def test_success(self):
        self.assertEqual(1, 1)

    def test_fail(self):
        self.assertEqual(2, 1)

    def test_error(self):
        self.assertEqual(1 / 0, 1)

Comments: Only one or zero exceptions (error or failure) need be reported because not more can be expected before tearDown. The package unittest expects that a second exception can be raised by tearDown. Therefore the lists errors and failures can contain only one or zero elements together before tearDown. Lines after "demo" comment are reporting a short result.

Demo output: (not important)

$ python3.5 -m unittest test

EF.
ERROR: test.MyTest.test_error
     ZeroDivisionError: division by zero
FAIL: test.MyTest.test_fail
     AssertionError: 2 != 1

==========================================================
... skipped usual output from unittest with tracebacks ...
...
Ran 3 tests in 0.002s

FAILED (failures=1, errors=1)

---

Comparision to other solutions - (with respect to commit history of Python source repository):

• This solution uses a private attribute of TestCase instance like many other solutions, but I checked carefully all relevant commits in the Python source repository that three alternative names cover the code history since Python 2.7 to 3.6.2 without any gap. It can be a problem after some new major Python release, but it could be clearly recognized, skipped and easily fixed later for a new Python. An advantage is that nothing is modified before running tearDown, it should never break the test and all functionality of unittest is supported, works with pytest and it could work many extending packages, but not with nosetest (not a suprise becase nosetest is not compatible e.g. with unittest.expectedFailure).

• The solutions with decorators on the user test methods or with a customized failureException (mgilson, Pavel Repin 2nd way, kenorb) are robust against future Python versions, but if everything should work completely, they would grow like a snow ball with more supported exceptions and more replicated internals of unittest. The decorated functions have less readable tracebacks (even more levels added by one decorator), they are more complicated for debugging and it is unpleassant if another more important decorator has a problem. (Thanks to mgilson the basic functionality is ready and known issues can be fixed.)

• The solution with modifired run method and catched result parameter

  • (scoffey) should work also for Python 2.6. The interpretation of results can be improved to requirements of the question, but nothing can work in Python 3.4+, because result is updated after tearDown call, never before.
  • Mark G.: (tested with Python 2.7, 3.2, 3.3, 3.4 and with nosetest)

• solution by exc_info() (Pavel Repin 2st way) works only with Python 2.

• Other solutions are principially similar, but less complete or with more disadvantages.

---

Explained by Python source repository
= Lib/unittest/case.py =
Python v 2.7 - 3.3

class TestCase(object):
    ...
    def run(self, result=None):
        ...
        self._outcomeForDoCleanups = result   # Python 3.2, 3.3
        # self._resultForDoCleanups = result  # Python 2.7
        #                                     # Python 2.6 - no result saved
        ...
        try:
            testMethod()
        except...   # many times for different exception classes
            result.add...(self, sys.exc_info())  # _addSkip, addError, addFailure
        ...
        try:
            self.tearDown()
        ...

Python v. 3.4 - 3.6

    def run(self, result=None):
        ...
        # outocome is a context manager to catch and collect different exceptions
        self._outcome = outcome  
        ...
        with outcome...(self):
            testMethod()
        ...
        with outcome...(self): 
            self.tearDown() 
        ... 
        self._feedErrorsToResult(result, outcome.errors)

Note (by reading Python commit messages): A reason why test results are so much decoupled from tests is memory leaks prevention. Every exception info can access to frames of the failed process state including all local variables. If a frame is assigned to a local variable in a code block that could also fail, then a cross memory refence could be easily created. It is not terrible, thanks to garbage collector, but the free memory can became fragmented more quickly than if the memory would be released correctly. This is a reason why exception information and traceback are converted very soon to strings and why temporary objects like self._outcome are encapsulated and are set to None in a finally block in order to memory leaks are prevented.