Tomcat Guice/JDBC Memory Leak

Question

I\'m experiencing a memory leak due to orphaned threads in Tomcat. Particularly, it seems that Guice and the JDBC driver are not closing threads.

Aug 8, 2012         


        

Answer 1

I've had the same issue, and as Jeff says, the "don't worry about it approach" was not the way to go.

I did a ServletContextListener that stops the hung thread when the context is being closed, and then registered such ContextListener on the web.xml file.

I already know that stopping a thread is not an elegant way to deal with them, but otherwise the server keeps on crashing after two or three deploys (it is not always possible to restart the app server).

The class I created is:

public class ContextFinalizer implements ServletContextListener {

    private static final Logger LOGGER = LoggerFactory.getLogger(ContextFinalizer.class);

    @Override
    public void contextInitialized(ServletContextEvent sce) {
    }

    @Override
    public void contextDestroyed(ServletContextEvent sce) {
        Enumeration<Driver> drivers = DriverManager.getDrivers();
        Driver d = null;
        while(drivers.hasMoreElements()) {
            try {
                d = drivers.nextElement();
                DriverManager.deregisterDriver(d);
                LOGGER.warn(String.format("Driver %s deregistered", d));
            } catch (SQLException ex) {
                LOGGER.warn(String.format("Error deregistering driver %s", d), ex);
            }
        }
        Set<Thread> threadSet = Thread.getAllStackTraces().keySet();
        Thread[] threadArray = threadSet.toArray(new Thread[threadSet.size()]);
        for(Thread t:threadArray) {
            if(t.getName().contains("Abandoned connection cleanup thread")) {
                synchronized(t) {
                    t.stop(); //don't complain, it works
                }
            }
        }
    }

}

After creating the class, then register it on the web.xml file:

<web-app...
    <listener>
        <listener-class>path.to.ContextFinalizer</listener-class>
    </listener>
</web-app>

Answer 2

See To prevent a memory leak, the JDBC Driver has been forcibly unregistered. Bill's answer deregisters all Driver instances as well as instances that may belong to other web applications. I have extended Bill's answer with a check that the Driver instance belongs to the right ClassLoader.

Here is the resulting code (in a separate method, because my contextDestroyed has other things to do):

// See https://stackoverflow.com/questions/25699985/the-web-application-appears-to-have-started-a-thread-named-abandoned-connect
// and
// https://stackoverflow.com/questions/3320400/to-prevent-a-memory-leak-the-jdbc-driver-has-been-forcibly-unregistered/23912257#23912257
private void avoidGarbageCollectionWarning()
{
    ClassLoader cl = Thread.currentThread().getContextClassLoader();
    Enumeration<Driver> drivers = DriverManager.getDrivers();
    Driver d = null;
    while (drivers.hasMoreElements()) {
        try {
            d = drivers.nextElement();
            if(d.getClass().getClassLoader() == cl) {
                DriverManager.deregisterDriver(d);
                logger.info(String.format("Driver %s deregistered", d));
            }
            else {
                logger.info(String.format("Driver %s not deregistered because it might be in use elsewhere", d.toString()));
            }
        }
        catch (SQLException ex) {
            logger.warning(String.format("Error deregistering driver %s, exception: %s", d.toString(), ex.toString()));
        }
    }
    try {
         AbandonedConnectionCleanupThread.shutdown();
    }
    catch (InterruptedException e) {
        logger.warning("SEVERE problem cleaning up: " + e.getMessage());
        e.printStackTrace();
    }
}

I wonder whether the call AbandonedConnectionCleanupThread.shutdown() is safe. Can it interfere with other web applications? I hope not, because the AbandonedConnectionCleanupThread.run() method is not static but the AbandonedConnectionCleanupThread.shutdown() method is.

Answer 3

I went a step further from Oso,improved the code above in two points:

1. Added the Finalizer thread to the need-to-kill check:

   for(Thread t:threadArray) {
           if(t.getName().contains("Abandoned connection cleanup thread") 
               ||  t.getName().matches("com\\.google.*Finalizer")
               ) {
           synchronized(t) {
               logger.warn("Forcibly stopping thread to avoid memory leak: " + t.getName());
               t.stop(); //don't complain, it works
           }
       }
   }
   

2. Sleep for a little while to give threads time to stop. Without that, tomcat kept complaining.

   try {
       Thread.sleep(1000);
   } catch (InterruptedException e) {
       logger.debug(e.getMessage(), e);
   }