TestNG retrying failed tests doesn't output the correct test results

Question

Setup: I have a class that extends the IRetryAnalyzer and have implemented a simple retry logic overriding the following method: public boolean retry(IT

Answer 1

My approach builds on Morvader's answer but adds the ability to define retry analyzers that adhere to the original intent of actually failing the test even if the method have passed after some retries.

I also didn't find a need to tend to the number of test in the onFinish() method, the numbers seemed fine in maven-surefire-plugin version 2.18

RetryListenerAdapter

public class RetryListenerAdapter extends TestListenerAdapter {

    @Override
    public void onTestFailure(ITestResult tr) {
        IRetryAnalyzer retryAnalyzer = tr.getMethod().getRetryAnalyzer();
        if (retryAnalyzer == null || !(retryAnalyzer instanceof IRetryAnalyzerWithSkip)) {
            super.onTestFailure(tr);
        } else if (((IRetryAnalyzerWithSkip) retryAnalyzer).isRetryable()) {
            tr.setStatus(ITestResult.SKIP);
            super.onTestSkipped(tr);
        } else {
            super.onTestFailure(tr);
        }
    }
}

IRetryAnalyzerWithSkip

public interface IRetryAnalyzerWithSkip extends IRetryAnalyzer {
    boolean isRetryable();
}

Retry

public class Retry implements IRetryAnalyzerWithSkip {
    private int retryCount = 0;
    private int maxRetryCount = 3;

    public boolean retry(ITestResult result) {

        if (retryCount < maxRetryCount) {
            retryCount++;
            return true;
        }
        return false;
    }

    @Override
    public boolean isRetryable() {
        return retryCount < maxRetryCount;
    }
}