Continuing test execution in junit4 even when one of the asserts fails

Question

I have my existing framework built up using Jfunc which provides a facility to continue exection even when one of the asserts in the test case fails. Jfunc uses junit 3.x framework. But now we are migrating to junit4 so I can't use Jfunc anymore and have replaced it with junit 4.10 jar.
Now the problem is since we have extensively used jfunc in our framework, and with junit 4 we want to make our code continue the execution even when one of the asserts fails in a test case.
Does anyone has any suggestion/idea for this, i know in junit the tests needs to be more atomic i.e. one assert per test case but we can't do that in our framework for some reason.

Answer 1

You can do this using an ErrorCollector rule.

To use it, first add the rule as a field in your test class:

public class MyTest {
    @Rule
    public ErrorCollector collector = new ErrorCollector();

    //...tests...
}

Then replace your asserts with calls to collector.checkThat(...).

e.g.

@Test
public void myTest() {
    collector.checkThat("a", equalTo("b"));
    collector.checkThat(1, equalTo(2));
}

Answer 2

I use the ErrorCollector too but also use assertThat and place them in a try catch block.

import static org.junit.Assert.*;
import static org.hamcrest.Matchers.*;

@Rule
public ErrorCollector collector = new ErrorCollector();

@Test
public void calculatedValueShouldEqualExpected() {
    try {
        assertThat(calculatedValue(), is(expected));
    } catch (Throwable t) {
        collector.addError(t);
        // do something
    }
}

Answer 3

You can also use assertj - soft assertion

@Test
public void testCollectErrors(){
   SoftAssertions softly = new SoftAssertions();
   softly.assertThat(true).isFalse();
   softly.assertThat(false).isTrue();
   // Don't forget to call SoftAssertions global verification !
   softly.assertAll();
}

Also exist other way to use it without manually invoke softly.assertAll();

1. with rule
2. with autoclosable
3. Using the static assertSoftly method

Answer 4

Use try/finally blocks. This worked in my case:

...
try {
    assert(...)
} finally {
    // code to be executed after assert
}
...

Answer 5

Try - catch, in "try" use the assertion, in "catch" add the possible error to collection. Then throw the exception at the end of test, in tearDown(). So if there will be fail/error in assert, it will be catched and test will continue. (The collection in example is static, you can also make new instance in setUp() for each @Test)

    public static List<String> errors = new ArrayList<>();


    try {
        //some assert...
    }
    catch (AssertionError error) {
        errors.add(error.toString());
    }



    @After
    public void tearDown() {

        try {
            if (!errors.isEmpty()) {
                throw new AssertionError(errors);
            }
        }
        finally {
            //empty list because it's static, alternatively make instance for each test in setUp()
            errors.clear();
        }
    }

Answer 6

Another option is the observable pattern in conjunction with lambda expressions. You can use something like the above.

public class MyTestClass {

    private final List<Consumer<MyTestClass>> AFTER_EVENT = new ArrayList<>();

    @After
    public void tearDown() {
        AFTER_EVENT.stream().forEach(c -> c.accept(this));
    }

    @Test
    public void testCase() {
        //=> Arrange
        AFTER_EVENT.add((o) -> {
            // do something after an assertion fail.
        }));

        //=> Act

        //=> Assert
        Assert.assertTrue(false);
    }
}