Has JUnit4 begun supporting ordering of test? Is it intentional?

问题

A newbie to JUnit (in fact JUnit 4) and came across suite way of executing test

@RunWith(Suite.class)
@Suite.SuiteClasses(
        {                               
                CreateNewProfile.class,
                EditProfile.class,

        })
public class ProfileTestSuite {

}

This is the code sample I came across while browsing through the test code base at my new employer. During execution I fund that - CreateNewProfile tests are executed first and then EditProfile, which does make sense but then it introduces dependency among tests.

I have been following non dependent test mechanism from couple of months (though I used to use TestNG and not JUnit) and would expect EditProfile being able to be executed in isolation as well. That is edit profile should take care of creating profile and then editing it and then asserting the operations.

My question here is - has Junit 4 introduced test ordering feature. Is this feature intended or one easter egg as I always felt JUnit = independent tests.

回答1:

No JUnit does not support test ordering, except in the manner that you say, through Suite. This only defines the order in which the test classes are executed. This has been there for a long time, including JUnit 3 Suite classes.

For a more complete explanation, there are three things we need to talk about here:

the ordering of tests classes within a test suite
the ordering of test classes when they are found by reflection by Eclipse or Maven
the ordering of test methods (annotated with @Test) within a test class.

The ordering of tests classes within a test suite

When you specify the list of classes to be executed in a test suite, you're defining an array, and these test classes will be executed in order, except when you're doing parallel execution. Unfortunately, this allows the introduction of dependencies between your test classes.

The ordering of test classes when they are found by reflection

When searching the classpath for classes, the order in which they are found is not guaranteed, so cannot be depended upon. It isn't actually JUnit which is doing the searching, but the Eclipse Junit plugin, or maven surefire or failsafe.

The ordering of test methods within a test class

JUnit does not guarantee the order of execution of tests within a class. Most of the time, on most JVMs pre version 7, the order in which they are found using reflection is in declaration order, ie the order they are in the file. This is the order in which they are executed. However, with JVM 7, this is no longer guaranteed, so there won't be a consistent order. There is a github issue #293 Sort test methods for predictability open with suggested solutions, and there is a thread on the junit mailing list: Alphabetizing test method run order?. So you cannot depend upon the order that tests will be executed using JUnit, but this is currently under discussion.

回答2:

No, ordering is not supported. And yes, it is intention:

KentBeck commented (December 30, 2010): Independent tests are more valuable, so JUnit by design does not support test ordering.

If you need this feature, you could use TestNG with it's @Test(sequential = true).

回答3:

@jeha answer is very good but he talks about test methods (the ones annotated with @Test) order. This order is indeed unspecified, probably because the order of methods in a class obtained via reflection is not guaranteed to correspond with the order of methods in the source file.

However from my experience they are executed in the same order they were defined most of the time.

You are asking about the order of test classes. This should not be a surprise: you are explicitly listing test classes using @Suite.SuiteClasses and JUnit probably has no business in shuffling them and running in different order. Are you concerned about introducing dependency between tests simply because you explicitly ordered them?

回答4:

You can extend the Suite runner:

public class MySuite extends Suite {

    public SuiteRunner(Class<?> klass, RunnerBuilder builder) throws InitializationError {
        super(klass, builder);
    }

    @Override
    protected List<Runner> getChildren() {
        List<Runner> children = super.getChildren();
        ... here modify the children list - you can remove or reorder test...
        return children;
    }
}

Then annotate your suite:

@RunWith(MySuite.class)
@Suite.SuiteClasses({                               
        CreateNewProfile.class,
        EditProfile.class})
public class ProfileTestSuite { }

The tests marked with @Ignore will be represented by IgnoredClassRunner and regular test will be represented by BlockJUnit4ClassRunner (or whatever runner you will annotate them - which will be probably extension of BlockJUnit4ClassRunner).

From the runner you can get the test class. Then you can use own annotations.

回答5:

JUnit 4.11 now supports specifying execution order using @FixMethodOrder annoation.

回答6:

Just bypass it by ordering the names of the class by using an _# before the class name. Ex _1testcaselogin _2testcaseverify

来源：https://stackoverflow.com/questions/7845929/has-junit4-begun-supporting-ordering-of-test-is-it-intentional

标签

java

junit

junit4