Lazy CSV Filtering / Parsing - Increasing Performance
Lazy Filtering CSV Files I had the need to filter through millions of log records, stored as numerous CSV files. The size of the records greatly exceeded my available memory so I wanted to go with a lazy approach. Java 8 Streams API With jdk8 we have the Streams API which paired with Apache commons-csv allows us to easily accomplish this. public class LazyFilterer { private static Iterable<CSVRecord> getIterable(String fileName) throws IOException { return CSVFormat .DEFAULT .withFirstRecordAsHeader() .parse(new BufferedReader(new FileReader(fileName))); } public static void main(String[] args