A useful metric for determining when the JVM is about to get into memory/GC trouble

Question

I have a scala data processing application that 95% of the time can handle the data thrown at it in memory. The remaining 5% if left unchecked doesn\'t usually hit OutOf

Answer 1

In addition to the MemoryMXBean notification mechanisms described in @Alla's link. You can use a combination of weak references and reference queues. This old but valid article has a good description of weak, soft, and phantom references and reference queues.

The basic idea is to create a large array (to reserve memory) create a weak or soft reference to it and when doing so, add it to a reference queue. When memory pressure triggers the collection of the weakly referenced array, you will get the reserve memory (breathing life into your application hopefully and giving it time). Have a thread polling the reference queue to determine when your reserve has been collected. You can then trigger the file streaming behavior of your application to finish the job. SoftReferences are more resilient to memory pressure than WeakReferences and make serve your purposes better.

Answer 2

may be this link will help you http://www.javaspecialists.eu/archive/Issue092.html

In my MemoryWarningSystem you add listeners that implement the MemoryWarningSystem.Listener interface, with one method memoryUsageLow(long usedMemory, long maxMemory) that will be called when the threshold is reached. In my experiments, the memory bean notifies us quite soon after the usage threshold has been exceeded, but I could not determine the granularity. Something to note is that the listener is being called by a special thread, called the Low Memory Detector thread, that is now part of the standard JVM.

What is the threshold? And which of the many pools should we monitor? The only sensible pool to monitor is the Tenured Generation (Old Space). When you set the size of the memory with -Xmx256m, you are setting the maximum memory to be used in the Tenured Generation.

Answer 3

One reliable way is to register a notification listener on GC events and check the memory health after all Full GC events. Directly after a full GC event, the memory used is your actual live set of data. If you at that point in time are low on free memory it is probably time start flusing to disk.

This way you can avoid false positives that often happens when you try to check memory with no knowledge of when a full GC has occurred, for example when using the MEMORY_THRESHOLD_EXCEEDED notification type.

You can register a notification listener and handle Full GC events using something like the following code:

// ... standard imports ommitted
import com.sun.management.GarbageCollectionNotificationInfo;

public static void installGCMonitoring() {
    List<GarbageCollectorMXBean> gcBeans = ManagementFactory.getGarbageCollectorMXBeans();
    for (GarbageCollectorMXBean gcBean : gcBeans) {
        NotificationEmitter emitter = (NotificationEmitter) gcBean;
        NotificationListener listener = notificationListener();
        emitter.addNotificationListener(listener, null, null);
    }
}

private static NotificationListener notificationListener() {
    return new NotificationListener() {
        @Override
        public void handleNotification(Notification notification, Object handback) {
            if (notification.getType()
                    .equals(GarbageCollectionNotificationInfo.GARBAGE_COLLECTION_NOTIFICATION)) {
                GarbageCollectionNotificationInfo info = GarbageCollectionNotificationInfo
                        .from((CompositeData) notification.getUserData());
                String gctype = info.getGcAction();
                if (gctype.contains("major")) {
                    // We are only interested in full (major) GCs
                    Map<String, MemoryUsage> mem = info.getGcInfo().getMemoryUsageAfterGc();
                    for (Entry<String, MemoryUsage> entry : mem.entrySet()) {
                        String memoryPoolName = entry.getKey();
                        MemoryUsage memdetail = entry.getValue();
                        long memMax = memdetail.getMax();
                        long memUsed = memdetail.getUsed();
                        // Use the memMax/memUsed of the pool you are interested in (probably old gen)
                        // to determine memory health.
                    }
                }
            }
        }
    };
}

Cred to this article where we first got this idea from.