Json Array reader file with spring batch

Question

I have a file as an input which contain a json Array :

[ {
  ...,
  ...
  },
  {
  ...,
  ...
  },
  {
  ...,
  ...
  }
]

I want to read it without breaking the spring batch principales (With the same way as FlatFileReader or XmlReader)

I didn't find any way to do it the readers already implemented in spring-batch .

What's the best way to implement this reader ?

Thanks in Advance

Answer 1

Assuming you want to model the StaxEventItemReader in that you want to read each item of the JSON array as an item in Spring Batch, here's what I'd recommend:

• RecordSeparatorPolicy - You'll need to implement your own RecordSepartorPolicy that indicates if you've finished reading in the full item or not. You can also use the RecordSeparatoerPolicy#postProcess to cleanup the beginning and ending [] you'll need to deal with as well as the comma delimiters.

• LineTokenizer - You'll then want to create your own LineTokenzier that parses JSON. I was just working on one today for a project so you can use that code as a start (consider it untested):

  public class JsonLineTokenizer implements LineTokenizer {
  
      @Override
      public FieldSet tokenize(String line) {
          List<String> tokens = new ArrayList<>();
  
          try {
              HashMap<String,Object> result =
                      new ObjectMapper().readValue(line, HashMap.class);
  
              tokens.add((String) result.get("field1"));
              tokens.add((String) result.get("field2")));
  
          } catch (IOException e) {
              throw new RuntimeException("Unable to parse json: " + line);
          }
  
          return new DefaultFieldSet(tokens.toArray(new String[0]), {"field1", "field2"});
      }
  }
  

Answer 2

This is the record separator policy I wrote starting from your suggestions and from the default implementation. I use an internal plain string representation for the read record, but i found out very simple to parse the JSON with codehaus jettison JSON object.

public class JsonRecordSeparatorPolicy extends SimpleRecordSeparatorPolicy {

/**
 * True if the line can be parsed to a JSON object.
 * 
 * @see RecordSeparatorPolicy#isEndOfRecord(String)
 */
@Override
public boolean isEndOfRecord(String line) {
    return StringUtils.countOccurrencesOf(line, "{") == StringUtils.countOccurrencesOf(line, "}")
            && (line.trim().endsWith("}") || line.trim().endsWith(",") || line.trim().endsWith("]") );
}

@Override
public String postProcess(String record) {
    if(record.startsWith("[")) record = record.substring(1);
    if(record.endsWith("]")) record = record.substring(0, record.length()-1);
    if(record.endsWith(",")) record = record.substring(0, record.length()-1);
    return super.postProcess(record);
}

}