A Nose plugin to specify the order of unit test execution

Question

I have a desire to use Nose for an over the wire integration test suite. However, the order of execution of some of these tests is important.

That said, I thought I would toss together a quick plugin to decorate a test with the order I want it executed: https://gist.github.com/Redsz/5736166

def Foo(unittest.TestCase):

    @step(number=1)
    def test_foo(self):
        pass

    @step(number=2)
    def test_boo(self):
        pass

From reviewing the built in plugins I had thought, I could simply override loadTestsFromTestCase and order the tests by the decorated 'step number'.:

def loadTestsFromTestCase(self, cls):
    """
    Return tests in this test case class. Ordered by the step definitions.
    """
    l = loader.TestLoader()
    tmp = l.loadTestsFromTestCase(cls)

    test_order = []
    for test in tmp._tests:
        order = test.test._testMethodName
        func = getattr(cls, test.test._testMethodName)
        if hasattr(func, 'number'):
            order = getattr(func, 'number')
        test_order.append((test, order))
    test_order.sort(key=lambda tup: tup[1])
    tmp._tests = (t[0] for t in test_order)
    return tmp

This method is returning the tests in the order I desire, however when the tests are being executed by nose they are not being executed in this order?

Perhaps I need to move this concept of ordering to a different location?

UPDATE: As per the comment I made, the plugin is actually working as expected. I was mistaken to trust the pycharm test reporter. The tests are running as expected. Rather than removing the question I figured I would leave it up.

Answer 1

From the documentation:

[...] nose runs functional tests in the order in which they appear in the module file. TestCase-derived tests and other test classes are run in alphabetical order.

So a simple solution might be to rename the tests in your test case:

class Foo(unittest.TestCase):

    def test_01_foo(self):
        pass

    def test_02_boo(self):
        pass

Answer 2

I found a solution for it using PyTest ordering plugin provided here.

Try py.test YourModuleName.py -vv in CLI and the test will run in the order they have appeared in your module (first test_foo and then test_bar)

I did the same thing and works fine for me.

Note: You need to install PyTest package and import it.