How to test if a file is “complete” (completely written) with Java

Question

Let\'s say you had an external process writing files to some directory, and you had a separate process periodically trying to read files from this directory. The problem to

Answer 1

You could use an external marker file. The writing process could create a file XYZ.lock before it starts creating file XYZ, and delete XYZ.lock after XYZ is completed. The reader would then easily know that it can consider a file complete only if the corresponding .lock file is not present.

Answer 2

One simple solution I've used in the past for this scenario with Windows is to use boolean File.renameTo(File) and attempt to move the original file to a separate staging folder:

boolean success = potentiallyIncompleteFile.renameTo(stagingAreaFile);

If success is false, then the potentiallyIncompleteFile is still being written to.

Answer 3

2 options that seems to solve this issue:

1. the best option- writer process notify reading process somehow that the writing was finished.
2. write the file to {id}.tmp, than when finish- rename it to {id}.java, and the reading process run only on *.java files. renaming taking much less time and the chance this 2 process work together decrease.

Answer 4

One more method to test that a file is completely written:

private void waitUntilIsReadable(File file) throws InterruptedException {
    boolean isReadable = false;
    int loopsNumber = 1;
    while (!isReadable && loopsNumber <= MAX_NUM_OF_WAITING_60) {
        try (InputStream in = new BufferedInputStream(new FileInputStream(file))) {
            log.trace("InputStream readable. Available: {}. File: '{}'",
                    in.available(), file.getAbsolutePath());
            isReadable = true;
        } catch (Exception e) {
            log.trace("InputStream is not readable yet. File: '{}'", file.getAbsolutePath());
            loopsNumber++;
            TimeUnit.MILLISECONDS.sleep(1000);
        }
    }
}

Answer 5

This possible to do by using Apache Commons IO maven library FileUtils.copyFile() method. If you try to copy file and get IOException its means that file is not completely saved.

Example:

public static void copyAndDeleteFile(File file, String destinationFile) {

    try {
        FileUtils.copyFile(file, new File(fileDirectory));
    } catch (IOException e) {
        e.printStackTrace();
        copyAndDeleteFile(file, fileDirectory, delayThreadPeriod);
    }

Or periodically check with some delay size of folder that contains this file:

FileUtils.sizeOfDirectory(folder);

Answer 6

First, there's Why doesn't OS X lock files like windows does when copying to a Samba share? but that's variation of what you're already doing.

As far as reading arbitrary files and looking for sizes, some files have that information, some do not, but even those that do do not have any common way of representing it. You would need specific information of each format, and manage them each independently.

If you absolutely must act on the file the "instant" it's done, then your writing process would need to send some kind of notification. Otherwise, you're pretty much stuck polling the files, and reading the directory is quite cheap in terms of I/O compared to reading random blocks from random files.