nose, unittest.TestCase and metaclass: auto-generated test_* methods not discovered

Question

This is a follow-up question for unittest and metaclass: automatic test_* method generation:

For this (fixed) unittest.TestCase layout:

#!/usr/bin/env python

import unittest


class TestMaker(type):

    def __new__(cls, name, bases, attrs):
        callables = dict([
            (meth_name, meth) for (meth_name, meth) in attrs.items() if
            meth_name.startswith('_test')
        ])

        for meth_name, meth in callables.items():
            assert callable(meth)
            _, _, testname = meth_name.partition('_test')

            # inject methods: test{testname}_v4,6(self)
            for suffix, arg in (('_false', False), ('_true', True)):
                testable_name = 'test{0}{1}'.format(testname, suffix)
                testable = lambda self, func=meth, arg=arg: func(self, arg)
                attrs[testable_name] = testable

        return type.__new__(cls, name, bases, attrs)


class TestCase(unittest.TestCase):

    __metaclass__ = TestMaker

    def test_normal(self):
        print 'Hello from ' + self.id()

    def _test_this(self, arg):
        print '[{0}] this: {1}'.format(self.id(), str(arg))

    def _test_that(self, arg):
        print '[{0}] that: {1}'.format(self.id(), str(arg))


if __name__ == '__main__':
    unittest.main()

This works using stdlib's framework. Expected and actual output:

C:\Users\santa4nt\Desktop>C:\Python27\python.exe test_meta.py
Hello from __main__.TestCase.test_normal
.[__main__.TestCase.test_that_false] that: False
.[__main__.TestCase.test_that_true] that: True
.[__main__.TestCase.test_this_false] this: False
.[__main__.TestCase.test_this_true] this: True
.
----------------------------------------------------------------------
Ran 5 tests in 0.015s

OK

However, since I am actually using nose, this trick seems to not agree with it. The output I got is:

C:\Users\santa4nt\Desktop>C:\Python27\python.exe C:\Python27\Scripts\nosetests test_meta.py
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

In short, the test_* methods generated by the metaclass do not register with nose. Can anyone shed a light on this?

Thanks,

Answer 1

So, after sleuthing through both stdlib's unittest and nose's loader and selector source code, it turns out that nose overrides unittest.TestLoader.getTestCaseNames to use its own selector (with plugin points).

Now, nose's selector looks for a potential method's method.__name__ to match certain regexes, black and white lists, and plugins' decisions.

In my case, the dynamically generated functions have its testable.__name__ == '<lambda>', matching none of nose's selector criteria.

To fix,

        # inject methods: test{testname}_v4,6(self)
        for suffix, arg in (('_false', False), ('_true', True)):
            testable_name = 'test{0}{1}'.format(testname, suffix)
            testable = lambda self, arg=arg: meth(self, arg)
            testable.__name__ = testable_name    # XXX: the fix
            attrs[testable_name] = testable

And sure enough:

(sandbox-2.7)bash-3.2$ nosetests -vv 
test_normal (test_testgen.TestCase) ... ok
test_that_false (test_testgen.TestCase) ... ok
test_that_true (test_testgen.TestCase) ... ok
test_this_false (test_testgen.TestCase) ... ok
test_this_true (test_testgen.TestCase) ... ok

----------------------------------------------------------------------
Ran 5 tests in 0.005s

OK