Spring Batch - Counting Processed Rows

Question

So I am creating a Spring Batch job for reading a CSV file and for certain rows which contain incomplete data; it checks, outputs to the log that the row is incomplete, and skips. It works great except at the end of the job I want it to log how many rows it found that were incomplete. Just something simple like "X incomplete rows were found".

I've Googled and searched around for a solution but not found anything really.

Any help is appreciated and any more info needed just ask.

Answer 1

Spring Batch itself keeps track of how many records it reads, writes, processes and how many it skips (for each of those numbers). That information is stored in the StepExecution. The StepExecution can be accessed from a StepExecutionListener. In this case an implementation of the afterStep method will suffice.

public class SkippedItemStepExecutionListener extends StepExecutionListenerSupport {

    @Override
    public ExitStatus afterStep(StepExecution stepExecution) {
        int skipped = stepExecution.getSkipCount(); // Total for read+write+process
        // Log it to somewhere.        
        return null;
    }
}

How to add it to your job/step is explained in the reference guide

Links

1. StepExecution javadoc
2. StepExecutionListener javadoc
3. Listener Configuration Reference

Answer 2

Manage to solve this, here's how I did it:

In the ItemProcessor I added an attribute and a method for getting access to the ExecutionContext from within the process method,

private ExecutionContext executionContext;

@BeforeStep
public void beforeStep(StepExecution stepExecution)
{
    this.executionContext = stepExecution.getExecutionContext();
}

...and then in the process() method when I find one of the rows I want to log, I can do this,

this.executionContext.putInt( "i_ThoseRows", this.executionContext.getInt( "i_ThoseRows", 0 ) + 1 );

Finally I add another method to the ItemProcessor to print the result at the end of the step,

@AfterStep
public void afterStep(StepExecution stepExecution)
{
    System.out.println( "Number of 'Those rows': " + this.executionContext.getInt( "i_ThoseRows", 0 ) );
}

Hope it helps someone

Answer 3

To complement @dogfight answer:

from spring batch docs:

The annotations are analysed by the XML parser for the elements, so all you need to do is use the XML namespace to register the listeners with a step

So to call the listener callback annotated functions beforeStep() and afterStep() you need to register you ItemProcessor as listener in the step:

<listeners>
    <listener ref="MyItemProcessor">
</listeners>

Otherwise you will have a NullPointerException when use the executionContext.