Moving files after failed validation (Java)

Question

We are validating XML files and depending on the result of the validation we have to move the file into a different folder.

When the XML is valid the validator returns a value and we can move the file without a problem. Same thing happens when the XML is not valid according to the schema.

If however the XML is not well formed the validator throws an exception and when we try to move the file, it fails. We believe there is still a handle in the memory somewhere that keeps hold of the file. We tried putting System.gc() before moving the file and that sorted the problem but we can't have System.gc() as a solution.

The code looks like this. We have a File object from which we create a StreamSource. The StreamSource is then passed to the validator. When the XML is not well formed it throws a SAXException. In the exception handling we use the .renameTo() method to move the file.

sc = new StreamSource(xmlFile);
validator.validate(sc);

In the catch we tried

validator.reset();
validator=null;
sc=null;

but still .renameTo() is not able to move the file. If we put System.gc() in the catch, the move will succeed.

Can someone enlight me how to sort this without System.gc()?

We use JAXP and saxon-9.1.0.8 as the parser.

Many thanks

Answer 1

Try creating a FileInputStream and passing that into StreamSource then close the FileInputStream when you're done. By passing in a File you have lost control of how/when to close the file handle.

Answer 2

When you set sc = null, you are indicating to the garbage collector that the StreamSource file is no longer being used, and that it can be collected. Streams close themselves in their destroy() method, so if they are garbage collected, they will be closed, and therefore can be moved on a Windows system (you will not have this problem on a Unix system).

To solve the problem without manually invoking the GC, simply call sc.getInputStream().close() before sc = null. This is good practice anyway.

A common pattern is to do a try .. finally block around any file handle usage, eg.

try {
    sc = new StreamSource(xmlFile);
    // check stuff
} finally {
    sc.getInputStream().close();
}
// move to the appropriate place

In Java 7, you can instead use the new try with resources block.

Answer 3

Try sc.getInputStream().close() in the catch

Answer 4

All the three answers already given are right : you must close the underlying stream, either with a direct call to StramSource, or getting getting the stream and closing it, or creating the stream yourself and closing it.

However, I've already seen this happening, under windows, since at least three years : even if you close the stream, really every stream, if you try to move or delete the file, it will throw exception .. unless ... you explicitly call System.gc().

However, since System.gc() is not mandatory for a JVM to actually execute a round of garbage collection, and since even if it was the JVM is not mandated to remove all possible garbage object, you have no real way of being sure that the file can be deleted "now".

I don't have a clear explanation, I can only imagine that probably the windows implementation of java.io somehow caches the file handle and does not close it, until the handle gets garbage collected.

It has been reported, but I haven't confirmed it, that java.nio is not subject to this behavior, cause it has more low level control on file descriptors.

A solution I've used in the past, but which is quite a hack, was to :

1. Put files to delete on a "list"
2. Have a background thread check that list periodically, calla System.gc and try to delete those files.
3. Remove from the list the files you managed to delete, and keep there those that are not yet ready to.

Usually the "lag" is in the order of a few milliseconds, with some exceptions of files surviving a bit more.

It could be a good idea to also call deleteOnExit on those files, so that if the JVM terminates before your thread finished cleaning some files, the JVM will try to delete them. However, deleteOnExit had it's own bug at the time, preventing exactly the removal of the file, so I didn't. Maybe today it's resolved and you can trust deleteOnExit.

This is the JRE bug that i find most annoying and stupid, and cannot believe it is still in existence, but unfortunately I hit it just a month ago on windows Vista with latest JRE installed.

Answer 5

Pretty old, but some people may still find this question.

1. I was using Oracle Java 1.8.0_77.
2. The problem occurs on Windows, not on Linux.
3. The StreamSource instanciated with a File seems to automatically allocate and release the file resource when processed by a validator or transformer. (getInputStream() returns null)
4. On Windows moving a file into the place of the source file (deleting the source file) after the processing is not possible.

Solution/Workaround: Move the file using

Files.move(from.toPath(), to.toPath(), REPLACE_EXISTING, ATOMIC_MOVE);

The use of ATOMIC_MOVE here is the critical point. Whatever the reason ist, it has something to do with the annoying behavior of Windows locking files.