Java: how to handle two process trying to modify the same file [duplicate]

Answer 1

So, you could use the method that Vineet Reynolds suggests in his comment.

If the two processes are actually just separate Threads within the same application, then you could set a flag somewhere to indicate that the file is open.

If it's two separate applications/processes altogether, the underlying filesystem should lock the files. When you get an I/O error from your output stream, you should be able to wrap a try/catch block around that, and then set your app up to retry later, or whatever the desired behavior is for your particular application.

Files aren't really designed to be written to simultaneously by multiple applications. If you can maybe describe why you want to write to a file simultaneously from multiple processes, there may be other solutions that could be suggested.

---

Updates after your recent edits: Ok, so you need at least 3 files to do what you're talking about. You definitely cannot try to read/write data to a single file concurrently. Your three files are:

1. the file that ProcessA dumps new/incoming data to
2. the file that ProcessB is currently working on
3. a final "output" file that holds the output from ProcessB.

ProcessB's loop:

• Take any data in file#2, process it, and write the output to file#3
• Delete file#2
• Repeat

ProcessA's loop:

• Write all new, incoming data to file#1
• Periodically check to see if file#2 exists
• When file#2 is deleted by ProcessB, then ProcessA should stop writing to file#1, rename file#1 to be file#2
• Start a new file#1
• Repeat

Answer 2

The code in the updated question is most likely that of process B, and not of process A. I'll assume that this is the case.

Considering that an instance of the OverlappingFileLockException exception is thrown, it appears that another thread in the same process is attempting to lock on the same file. This is not a conflict between A and B, but rather a conflict within B, if one goes by the API documentation on the lock() method and when the condition under which it throws OverlappingFileLockException:

If a lock that overlaps the requested region is already held by this Java virtual machine, or if another thread is already blocked in this method and is attempting to lock an overlapping region of the same file

The only solution to prevent this, is to have any other thread in B prevented from acquiring a lock on the same file, or the same overlapping region in the file.

The IOException being thrown has a bit more interesting message. It probably confirms the above theory, but without looking at the entire source code, I cannot confirm anything. The lock method is expected to block until the exclusive lock is acquired. If it was acquired, then there ought to be no problem in writing to the file. Except for one condition. If the file has already been opened (and locked) by the same JVM in a different thread, using a File object (or in other words, a second/different file descriptor), then the attempted write on the first file descriptor will fail even if the lock was acquired (after all, the lock does not lock out other threads).

An improved design, would be to have a single thread in each process that acquires an exclusive lock on the file (while using a single File object, or a single file descriptor) for only a certain amount of time, perform the required activity in the file, and then release the lock.

Answer 3

If this is two separate applications trying to access the file. The one would through an IOException because he cant access it. If that occurs, in the catch(IOException err){} add code to pause the current thread for a few milliseconds and then recursively try to write again - until it gains access.

public boolean writeFile()
{
    try
    {
       //write to file here
        return true;
    }
    catch (IOException err) // Can't access
    {
        try
        {
            Thread.sleep(200); // Sleep a bit
            return writeFile(); // Try again
        }
        catch (InterruptedException err2)
        {
           return writeFile(); // Could not sleep, try again anyway
        }
    }
}

This will keep on trying until you get a StackOverflow Exception meaning that it went in too deep; but the chance of that happening in this situation is very little - would only happen if the file was to be kept open for really long time by other application.

Hope this helps!

Answer 4

Think about this using the MapReduce mentality. Let's assume the each program is writing output without reading the other's output. I would write two separate files, and then have a 'reduce' phase. Your reduction might be a simple chronologically ordered merge.

If, however, your programs' require one-anothers' output. You have a very different problem and need to rethink how you are partitioning the work.

Finally, if the two programs' outputs are similar but independent, and you are writing it into one file so a third program can read it all, consider changing the third program to read both files.