How do you flush a buffered log4j FileAppender?

Question

In log4j, when using a FileAppender with BufferedIO=true and BufferSize=xxx properties (i.e. buffering is enabled), I want to be able to flush the log during normal shutdown

Answer 1

public static void flushAllLogs()
{
    try
    {
        Set<FileAppender> flushedFileAppenders = new HashSet<FileAppender>();
        Enumeration currentLoggers = LogManager.getLoggerRepository().getCurrentLoggers();
        while(currentLoggers.hasMoreElements())
        {
            Object nextLogger = currentLoggers.nextElement();
            if(nextLogger instanceof Logger)
            {
                Logger currentLogger = (Logger) nextLogger;
                Enumeration allAppenders = currentLogger.getAllAppenders();
                while(allAppenders.hasMoreElements())
                {
                    Object nextElement = allAppenders.nextElement();
                    if(nextElement instanceof FileAppender)
                    {
                        FileAppender fileAppender = (FileAppender) nextElement;
                        if(!flushedFileAppenders.contains(fileAppender) && !fileAppender.getImmediateFlush())
                        {
                            flushedFileAppenders.add(fileAppender);
                            //log.info("Appender "+fileAppender.getName()+" is not doing immediateFlush ");
                            fileAppender.setImmediateFlush(true);
                            currentLogger.info("FLUSH");
                            fileAppender.setImmediateFlush(false);
                        }
                        else
                        {
                            //log.info("fileAppender"+fileAppender.getName()+" is doing immediateFlush");
                        }
                    }
                }
            }
        }
    }
    catch(RuntimeException e)
    {
        log.error("Failed flushing logs",e);
    }
}

Answer 2

Managed to answer my own question :-)

When shutting down the LogManager:

LogManager.shutdown();

all buffered logs get flushed.

Answer 3

Sharing my experience with using "Andrey Kurilov"'s code example, or at least simmilar.

What I actually wanted to achieve was to implement an asynchronous log entries with manual flush (immediateFlush = false) to ensure an idle buffers content will be flushed before the bufferSize is reached.

The initial performance results were actually comparable with the ones achieved with the AsyncAppender - so I think it is a good alternative of it.

The AsyncAppender is using a separate thread (and additional dependency to disruptor jar), which makes it more performant, but with the cost of more CPU and even more disk flushing(no matter with high load flushes are made on batches).

So if you want to save disk IO operations and CPU load, but still want to ensure your buffers will be flushed asynchronously at some point, that is the way to go.

Answer 4

Maybe you could override WriterAppender#shouldFlush( LoggingEvent ), so it would return true for a special logging category, like log4j.flush.now, and then you call:

LoggerFactory.getLogger("log4j.flush.now").info("Flush")

http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/WriterAppender.html#shouldFlush%28org.apache.log4j.spi.LoggingEvent%29

Answer 5

Try:

LogFactory.releaseAll();

Answer 6

I have written an appender that fixes this, see GitHub or use name.wramner.log4j:FlushAppender in Maven. It can be configured to flush on events with high severity and it can make the appenders unbuffered when it receives a specific message, for example "Shutting down". Check the unit tests for configuration examples. It is free, of course.