How to change the format of logged messages temporarily, in Python?

Question

What is the simplest method for temporarily changing the logging message format, in Python (through the logging module)?

The goal is to have some standard message fo

Answer 1

There are several ways. Apart from the already documented ones (extra argument to logging calls, LoggerAdapter, Filter) , another way would be to specify a custom formatting class, whose instance you can keep informed about the file being processed. For example:

class FileProcessingFormatter(logging.Formatter):
    def __init__(self, fmt, datefmt=None, current_file=None):
        super(FileProcessingFormatter, self).__init__(fmt, datefmt)
        self.orig_fmt = fmt
        self.current_file = current_file

    def format(self, record):
        if self.current_file is None:
            self._fmt = self.orig_fmt.replace('__FILE_PLACEHOLDER__', '')
        else:
            self._fmt = self.orig_fmt.replace('__FILE_PLACEHOLDER__',
                            ' while processing %r' % self.current_file)
        return super(FileProcessingFormatter, self).format(record)

Instantiate the formatter ...

f = FileProcessingFormatter('%(levelname)s__FILE_PLACEHOLDER__ %(message)s')
for h in relevant_handlers:
    h.setFormatter(f)

Process files ...

f.current_file = fn
process_file(fn)
f.current_file = None

This is very simplistic - for example, not for use in threaded environments if file processing is done by different threads concurrently.

Update: Although the root logger's handlers are accessible via logging.getLogger().handlers, this is an implementation detail that could change. As your requirement is not that basic, you could perhaps use dictConfig() to configure your logging (available via the logutils project for older versions of Python).

Answer 2

if you want to dynamic change format of log. it could be made like this.

logger = logging.getLogger()
# Change format of handler for the logger
logger.handlers[0].setFormatter(logging.Formatter('%(asctime)s,%(msecs)d %(name)s %(levelname)s %(message)s'))
# Print log
logging.info(log)
# return other format
logger.handlers[0].setFormatter(logging.Formatter('%(message)s'))

Answer 3

I don't recommend this; but you can say assume the first root handler is the one that's screwed up and modify it directly

import logging
ROOT_LOGGER = logging.getLogger()
ROOT_LOGGER.handlers[0].setFormatter(logging.Formatter(
    '%(asctime)s:%(levelname)s:%(name)s:%(message)s\n'
))

if you are in any system with managed logging; this is probably going to shoot your foot; it really would be best to be able to determine an exact reference to the handler you want to modify and modify that;

but nobody cares how broken it is if it works right?/s

Answer 4

Here is a simple solution, that can be deduced from Vinay Sajip's own HOWTO; it basically updates the logging formatter with setFormatter():

import logging

logger = logging.getLogger()  # Logger
logger_handler = logging.StreamHandler()  # Handler for the logger
logger.addHandler(logger_handler)

# First, generic formatter:
logger_handler.setFormatter(logging.Formatter('%(message)s'))
logger.error('error message')  # Test

# New formatter for the handler:
logger_handler.setFormatter(logging.Formatter('PROCESSING FILE xxx - %(message)s'))
logger.error('error message')  # Test

This correctly produces:

error message
PROCESSING FILE xxx - error message

(where xxx can be set dynamically to the file being processed, as asked for in the question).