pytest running scenarios in the correct order in the class

Question

So I have the following structure:

class Test(object):

   def test_1(self):
      pass

   def test_2(self):
      pass

   def test_3(self):
      pass

it runs great, NOW I'm adding the "scenarios" (as it's recommended at pytest - A quick port of “testscenarios”):

def pytest_generate_tests(metafunc):
    idlist = []
    argvalues = []
    for scenario in metafunc.cls.scenarios:
        idlist.append(scenario[0])
        items = scenario[1].items()
        argnames = [x[0] for x in items]
        argvalues.append(([x[1] for x in items]))
    metafunc.parametrize(argnames, argvalues, ids=idlist)

class Test(object):
       scenarios = ['1' {'arg':'value1'},
                    '2' {'arg':'value2'}]

       def test_1(self, arg):
          pass

       def test_2(self, arg):
          pass

       def test_3(self, arg):
          pass

When I run it the ORDER of tests is wrong, I get:

test_1[1]  
test_1[2]  
test_2[1]   
test_2[2]  
test_3[1]  
test_3[2]

Doesn't really look like a scenario for the Test class.

QUESTION: Is the any solution to run it in the correct order? like:

test_1[1]
test_2[1]
test_3[1]
test_1[2]
test_2[2]
test_3[2]

Answer 1

The upcoming pytest-2.3 has support for better (resource-based) ordering, and i just updated the scenario example in the docs: https://docs.pytest.org/en/latest/example/parametrize.html#a-quick-port-of-testscenarios

You can preliminary install the current development version with

pip install -i http://pypi.testrun.org -U pytest

and should get pytest-2.3.0.dev15 with "py.test --version" and be able to use it.

Answer 2

py.test runs tests in a distributed fashion, which means the order is essentially random.

You should use the -n option and set the process number to 1. Then tests should be run in alphabetical order by the single process spawned.

More than this I don't know if you can do. Anyway depending on the order of tests is generally bad design. So you should try to not depend on it at all.