Tracking down a memory leak / garbage-collection issue in Java

Question

This is a problem I have been trying to track down for a couple months now. I have a java app running that processes xml feeds and stores the result in a database. There hav

Answer 1

I had the same problem, with couple of differences..

My technology is the following:

grails 2.2.4

tomcat7

quartz-plugin 1.0

I use two datasources on my application. That is a particularity determinant to bug causes..

Another thing to consider is that quartz-plugin, inject hibernate session in quartz threads, just like @liam says, and quartz threads still alive, untill I finish application.

My problem was a bug on grails ORM combined with the way the plugin handle session and my two datasources.

Quartz plugin had a listener to init and destroy hibernate sessions

public class SessionBinderJobListener extends JobListenerSupport {

    public static final String NAME = "sessionBinderListener";

    private PersistenceContextInterceptor persistenceInterceptor;

    public String getName() {
        return NAME;
    }

    public PersistenceContextInterceptor getPersistenceInterceptor() {
        return persistenceInterceptor;
    }

    public void setPersistenceInterceptor(PersistenceContextInterceptor persistenceInterceptor) {
        this.persistenceInterceptor = persistenceInterceptor;
    }

    public void jobToBeExecuted(JobExecutionContext context) {
        if (persistenceInterceptor != null) {
            persistenceInterceptor.init();
        }
    }

    public void jobWasExecuted(JobExecutionContext context, JobExecutionException exception) {
        if (persistenceInterceptor != null) {
            persistenceInterceptor.flush();
            persistenceInterceptor.destroy();
        }
    }
}

In my case, persistenceInterceptor instances AggregatePersistenceContextInterceptor, and it had a List of HibernatePersistenceContextInterceptor. One for each datasource.

Every opertion do with AggregatePersistenceContextInterceptor its passed to HibernatePersistence, without any modification or treatments.

When we calls init() on HibernatePersistenceContextInterceptor he increment the static variable below

private static ThreadLocal nestingCount = new ThreadLocal();

I don't know the pourpose of that static count. I just know he it's incremented two times, one per datasource, because of the AggregatePersistence implementation.

Until here I just explain the cenario.

The problem comes now...

When my quartz job finish, the plugin calls the listener to flush and destroy hibernate sessions, like you can see in source code of SessionBinderJobListener.

The flush occurs perfectly, but the destroy not, because HibernatePersistence, do one validation before close hibernate session... It examines nestingCount to see if the value is grather than 1. If the answer is yes, he not close the session.

Simplifying what was did by Hibernate:

if(--nestingCount.getValue() > 0)
    do nothing;
else
    close the session;

That's the base of my memory leak.. Quartz threads still alive with all objects used in session, because grails ORM not close session, because of a bug caused because I have two datasources.

To solve that, I customize the listener, to call clear before destroy, and call destroy two times, (one for each datasource). Ensuring my session was clear and destroyed, and if the destroy fails, he was clear at least.