pytest running scenarios in the correct order in the class

问题:

So I have the following structure:

class Test(object):     def test_1(self):       pass     def test_2(self):       pass     def test_3(self):       pass 

it runs great, NOW I'm adding the "scenarios" (as it's recommended at pytest - A quick port of “testscenarios”):

def pytest_generate_tests(metafunc):     idlist = []     argvalues = []     for scenario in metafunc.cls.scenarios:         idlist.append(scenario[0])         items = scenario[1].items()         argnames = [x[0] for x in items]         argvalues.append(([x[1] for x in items]))     metafunc.parametrize(argnames, argvalues, ids=idlist)  class Test(object):        scenarios = ['1' {'arg':'value1'},                     '2' {'arg':'value2'}]         def test_1(self, arg):           pass         def test_2(self, arg):           pass         def test_3(self, arg):           pass 

When I run it the ORDER of tests is wrong, I get:

test_1[1]   test_1[2]   test_2[1]    test_2[2]   test_3[1]   test_3[2] 

Doesn't really look like a scenario for the Test class.

QUESTION: Is the any solution to run it in the correct order? like:

test_1[1] test_2[1] test_3[1] test_1[2] test_2[2] test_3[2] 

回答1:

The upcoming pytest-2.3 has support for better (resource-based) ordering, and i just updated the scenario example in the docs: https://docs.pytest.org/en/latest/example/parametrize.html#a-quick-port-of-testscenarios

You can preliminary install the current development version with

pip install -i http://pypi.testrun.org -U pytest 

and should get pytest-2.3.0.dev15 with "py.test --version" and be able to use it.

回答2:

py.test runs tests in a distributed fashion, which means the order is essentially random.

You should use the -n option and set the process number to 1. Then tests should be run in alphabetical order by the single process spawned.

More than this I don't know if you can do. Anyway depending on the order of tests is generally bad design. So you should try to not depend on it at all.

文章来源: pytest running scenarios in the correct order in the class