I am building a process to merge a few big sorted csv files. I am currently looking into using Univocity to do this. The way I setup the merge is to us
There are two approaches to read iteratively instead of loading everything in memory, the first one is to use a BeanProcessor
instead of BeanListProcessor
:
settings.setRowProcessor(new BeanProcessor(Address.class) {
@Override
public void beanProcessed(Address address, ParsingContext context) {
// your code to process the each parsed object here!
}
To read beans iteratively without a callback (and to perform some other common processes), we created a CsvRoutines class (which extends from AbstractRoutines - more examples here):
File input = new File("/path/to/your.csv")
CsvParserSettings parserSettings = new CsvParserSettings();
//...configure the parser
// You can also use TSV and Fixed-width routines
CsvRoutines routines = new CsvRoutines(parserSettings);
for (Address address : routines.iterate(Address.class, input, "UTF-8")) {
//process your bean
}
Hope this helps!