How to list a 2 million files directory in java without having an “out of memory” exception

Question

I have to deal with a directory of about 2 million xml\'s to be processed.

I\'ve already solved the processing distributing the work between machines and threads us

Answer 1

This also requires Java 7, but it's simpler than the Files.walkFileTree answer if you just want to list the contents of a directory and not walk the whole tree:

Path dir = Paths.get("/some/directory");
try (DirectoryStream stream = Files.newDirectoryStream(dir)) {
    for (Path path : stream) {
        handleFile(path.toFile());
    }
} catch (IOException e) {
    handleException(e);
}

The implementation of DirectoryStream is platform-specific and never calls File.list or anything like it, instead using the Unix or Windows system calls that iterate over a directory one entry at a time.