unittest.py doesn't play well with trace.py - why?

Question

Wow. I found out tonight that Python unit tests written using the unittest module don\'t play well with coverage analysis under the trace module.

Answer 1

I don't know why trace doesn't work properly, but coverage.py does:

$ coverage run foobar.py
.
----------------------------------------------------------------------
Ran 1 test in 0.001s

OK
$ coverage report
Name     Stmts   Miss  Cover
----------------------------
foobar       6      0   100%

Answer 2

I like Theran's answer but there were some catches with it, on Python 3.6 at least:

if I ran foobar.py that went fine, but if I ran foobar.py Sometestclass, to execute only Sometestclass, trace did not pick that up and ran all tests anyway.

My workaround was to specify defaultTest, when appropriate:

remember that unittest usually are run as

python foobar.py <-flags and options> <TestClass.testmethod> so targeted test is always the last arg, unless it's a unittest option, in which case it starts with -. or it's the foobar.py file itself.

    lastarg = sys.argv[-1]
    #not a flag, not foobar.py either...
    if not lastarg.startswith("-") and not lastarg.endswith(".py"):
        defaultTest = lastarg
    else:
        defaultTest = None

    unittest.main(module=os.path.splitext(os.path.basename(__file__))[0], defaultTest=defaultTest)

anyway, now trace only executes the desired tests, or all of them if I don't specify otherwise.

Answer 3

A simpler workaround is to pass the name of the module explicitly to unittest.main:

import unittest

class Tester(unittest.TestCase):
    def test_true(self):
        self.assertTrue(True)

if __name__ == "__main__":
    unittest.main(module='foobar')

trace messes up test discovery in unittest because of how trace loads the module it is running. trace reads the module source code, compiles it, and executes it in a context with a __name__ global set to '__main__'. This is enough to make most modules behave as if they were called as the main module, but doesn't actually change the module which is registered as __main__ in the Python interpreter. When unittest asks for the __main__ module to scan for test cases, it actually gets the trace module called from the command line, which of course doesn't contain the unit tests.

coverage.py takes a different approach of actually replacing which module is called __main__ in sys.modules.