Don't start nested test cases when outer test case fails with JUnit 5

Question

I have the following test class:

import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;
import static org.junit.jupiter.api.Assertions.assertTrue;

public class HierarchicalTest {

    @Test
    void checkOuter() {
        assertTrue(false);
    }

    @Nested
    @TestInstance(TestInstance.Lifecycle.PER_CLASS)
    class PrepBTest {

        @Test
        void checkInnerA() {}

        @Test
        void checkInnerB() {}
    }
}

I want to have the behavior that checkInnerA() and checkInnerB() won't be executed when checkOuter() fails.

On the other side checkInnerB() should be executed when checkInnerA() fails because it is on the same level.

Is there a simple soulution (e.g. with JUnit 5 extension) to achieve this behavior? In my opinion that's often the behavior which is wished.

Answer 1

UPDATE:

As of JUnit Jupiter 5.4, you can develop an extension that implements the TestWatcher and ExecutionCondition APIs to achieve this behavior.

In the testFailed() method from the TestWatcher API you need to track test classes that have failures, and you need to store this information in the root ExtensionContext.Store.

In the evaluateExecutionCondition() method from the ExecutionCondition API you need to determine if the current class is a @Nested test class (i.e., an inner class) and check if the enclosing test class had failures. If that holds true, you need to disable the current @Nested test class and otherwise enable it.

Those are the general guidelines. For a working example, please see the SkipOnFailuresInEnclosingClassExtension I just posted to my junit5-demo repository on GitHub. That example goes one step further by only skipping @Nested test classes if they are also annotated with @SkipOnFailuresInEnclosingClass. The OuterTests class shows the annotation and extension in action.

---

No, as of JUnit Jupiter 5.3, there is currently no way to achieve that with out-of-the-box solutions.

You could potentially write a custom extension that tracks the success of tests in an enclosing test class -- for example, by implementing TestExecutionExceptionHandler. That would need to be stored in the ExtensionContext.Store. The extension would then need to implement ExecutionCondition to programmatically disable nested test classes.

It's unfortunately not very straightforward to track the "success" of previously executed tests currently, but that should improve with the introduction of the new TestWatcher extension API that is currently slated for inclusion in the upcoming JUnit Jupiter 5.4: https://github.com/junit-team/junit5/issues/542