Spring Batch: How to setup a FlatFileItemReader to read a json file?

Question

My approach so far:

@Bean
FlatFileItemReader<Blub> flatFileItemReader() {
    FlatFileItemReader<Blub> reader = new FlatFileItemReader<>();

    reader.setResource(new FileSystemResource("test.json"));

    JsonLineMapper lineMapper = new JsonLineMapper();

    reader.setLineMapper(lineMapper);

    return reader;
}

The challenge is: reader.setLineMapper() cannot use the JsonLineMapper. How to use the JsonLineMapper properly?

Answer 1

How to setup a FlatFileItemReader to read a json file?

It depends on the format of your json file:

1. Each line is a json object (known as NDJson)

For example:

{object1}
{object2}

then you have two options:

• 1.1 Use the JsonLineMapper which returns a Map<String, Object>. In this case, your reader should also return Map<String, Object> and you can use an item processor to transform items from Map<String, Object> to Blub (BTW, transforming data from one type to another is a typical use case for an item processor)
• 1.2 Use a custom implementation of LineMapper<Blub> based on Jackson or Gson or any other library (as shown in the answer by @clevertension)

2. Lines are wrapped in a json array

For example:

[
 {object1},
 {object2}
]

then you can use the new JsonItemReader that we introduced in version 4.1.0.M1 (See example in the blog post here: https://spring.io/blog/2018/05/31/spring-batch-4-1-0-m1-released#add-a-new-json-item-reader).

There are similar questions to this one, I'm adding them here for reference:

• How to read a complex JSON in spring batch?
• Json Array reader file with spring batch

Answer 2

create a class BlubJsonLineMapper

public class BlubJsonLineMapper implements LineMapper<Blub> {

    private ObjectMapper mapper = new ObjectMapper();


    /**
     * Interpret the line as a Json object and create a Blub Entity from it.
     * 
     * @see LineMapper#mapLine(String, int)
     */
    @Override
    public Blub mapLine(String line, int lineNumber) throws Exception {
        return mapper.readValue(line, Blub.class);
    }

}

then you can set in the FlatFileItemReader

@Bean
FlatFileItemReader<Blub> flatFileItemReader() {
    FlatFileItemReader<Blub> reader = new FlatFileItemReader<>();

    reader.setResource(new FileSystemResource("test.json"));

    BlubJsonLineMapper lineMapper = new BlubJsonLineMapper();

    reader.setLineMapper(lineMapper);

    return reader;
}

Answer 3

I have build a small demo for Json. If you need any more than it, let me know I can build another example for you

https://github.com/bigzidane/spring-batch-jsonListItem-reader