What can cause Java to keep running after System.exit()?

Question

I have a Java program which is being started via ProcessBuilder from another Java program. System.exit(0) is called from the child program, but for some of our users (on Windows) the java.exe process associated with the child doesn't terminate. The child program has no shutdown hooks, nor does it have a SecurityManager which might stop System.exit() from terminating the VM. I can't reproduce the problem myself on Linux or Windows Vista. So far, the only reports of the problem come from two Windows XP users and one Vista user, using two different JREs (1.6.0_15 and 1.6.0_18), but they're able to reproduce the problem every time.

Can anyone suggest reasons why the JVM would fail to terminate after System.exit(), and then only on some machines?

Edit 1: I got the user to install the JDK so we could get a thread dump from the offending VM. What the user told me is that the VM process disappears from VisualVM as soon as he clicks on the 'Quit' item in my menu---but, according to Windows Task Manager, the process hasn't terminated, and no matter how long the user waits (minutes, hours), it never terminates.

Edit 2: I have confirmed now that Process.waitFor() in the parent program never returns for at least one of the users having the problem. So, to summarize: The child VM seems to be dead (VisualVM doesn't even see it) but the parent still sees the process as live and so does Windows.

Answer 1

The parent process has one thread dedicated to consuming each of the child's STDOUT and STDERR (which passes that output through to a log file). So far as I can see, those are working properly, since we're seeing all the output we expect to see in the log

i had a similar problem with my program not disappearing from task mgr when i was consuming the stdout/stderr. in my case, if I closed the stream that was listening before calling system.exit() then the javaw.exe hung around. strange, it wasn't writing to the stream...

the solution in my case was to simply flush the stream rather than close it before existing. of course, you could always flush and then redirect back to stdout and stderr before exit.

Answer 2

This can happen if your code (or a library you use) has a shutdown hook or a finalizer that doesn't finish cleanly.

A more vigorous (so should only be used in extreme cases!) way to force shutdown is by running:

Runtime.getRuntime().halt(0);

Answer 3

Here are a couple of scenarios...

Per the definition of a Thread in http://java.sun.com/j2se/1.4.2/docs/api/java/lang/Thread.html

...

When a Java Virtual Machine starts up, there is usually a single non-daemon thread (which typically calls the method named main of some designated class). The Java Virtual Machine continues to execute threads until either of the following occurs:

1) The exit method of class Runtime has been called and the security manager has permitted the exit operation to take place. 2) All threads that are not daemon threads have died, either by returning from the call to the run method or by throwing an exception that propagates beyond the run method.

Another possibility is if the method runFinalizersOnExit has been called. as per the documentation in http://java.sun.com/j2se/1.4.2/docs/api/java/lang/System.html Deprecated. This method is inherently unsafe. It may result in finalizers being called on live objects while other threads are concurrently manipulating those objects, resulting in erratic behavior or deadlock. Enable or disable finalization on exit; doing so specifies that the finalizers of all objects that have finalizers that have not yet been automatically invoked are to be run before the Java runtime exits. By default, finalization on exit is disabled. If there is a security manager, its checkExit method is first called with 0 as its argument to ensure the exit is allowed. This could result in a SecurityException.

Answer 4

Maybe a badly written finalizer? A shutdown hook was my first thought when I read the subject line. Speculation: would a thread that catches InterruptedException and keeps on running anyway hold up the exit process?

It seems to me that if the problem is reproducible, you should be able to attach to the JVM and get a thread list/stack trace that shows what is hung up.

Are you sure that the child is still really running and that it's not just an unreaped zombie process?

Answer 5

Does the parent process consumes the error- and outputstream from the child process? If under some OS the childprocess print out some errors/warning on stdout/stderr and the parent process is not consuming the streams, the childprocess will block and not reach System.exit();

Answer 6

Hy i had the same problem, but the couse for me was i was using the remote Debugging(VmArgs: -Xdebug -Xrunjdwp:transport=dt_socket,address=%port%,server=y,suspend=y) when i disabled this the java.exe Process exited as expected.

Answer 7

Check if there is a deadlock.

For example,

Runtime.getRuntime().addShutdownHook(new Thread(this::close));
System.exit(1);

and in the close()

public void close() {
// some code
System.exit(1);
}

The System.exit() actually calls the shutdown hook and finalizers. So, if your shutdown hook calls the exit() again internally, for some reason (for example, when the close() raises an exception for which you want to exit the program, then you will call exit() there also). Like this..

public void close() {
  try {
  // some code that raises exception which requires to exit the program
  } catch(Exception exceptionWhichWhenOccurredShouldExitProgram) {
    log.error(exceptionWhichWhenOccurredShouldExitProgram);
    System.exit(1);
  }
}

Though, it is a good practice to throw the exception, some may choose to log and exit.

Note, also, that Ctrl+C will also not work if there is a deadlock. Since it also calls the shutdown hook.

Anyways, if it is the case, the problem can be solved by this workaround:

private static AtomicBoolean exitCalled=new AtomicBoolean();

    private static void exit(int status) {
        if(!exitCalled.get()) {
            exitCalled.set(true);
            System.exit(status);
        }
    }

    Runtime.getRuntime().addShutdownHook(new Thread(MyClass::close));
      exit(1);
    }

    private static void close() {
        exit(1);
    }

P.S: I feel that the above exit() version must actually be written in the System.exit() method only (may be some PR for JDK?) Because, there is practically no point (at least from what I see) in entertaining a deadlock in System.exit()

Answer 8

I think all of the obvious causes have been provisionally covered; e.g. finalizers, shutdown hooks, not correctly draining standard output / standard error in the parent process. You now need more evidence to figure what is going on.

Suggestions:

1. Set up a Windows XP or Vista machine (or virtual), install the relevant JRE and your app, and try to reproduce the problem. Once you can reproduce the problem, either attach a debugger or send the relevant signal to get a thread dump to standard error.

2. If you cannot reproduce the problem as above, get one of your users to take a thread dump, and forward you the log file.

Answer 9

Another case not mentioned here is if the shutdown hooks hang on something so be careful in writing the shutdown hook code(and if a 3rd party library registered a shutdown hook that is hanging as well).

Dean