How to change filehandle with Python logging on the fly with different classes and imports

Question

I cannot perform an on-the-fly logging fileHandle change.

For example, I have 3 classes

one.py

import logging
class One():
    d         


        

Answer 1

I found an easier way than the above 'accepted' answer. If you have a reference to the handler, all you need to do is call the close() method and then set the baseFilename property. When you assign baseFilename, be sure to use os.path.abspath(). There's a comment in the library source that indicates it's needed. I keep my configuration stuff in a global dict() so it's easy to keep the FileHandler reference objects. As you can see below, it only takes 2 lines of code to change a log filename for a handler on the fly.

import logging

def setup_logging():
  global config

  if config['LOGGING_SET']:
    config['LOG_FILE_HDL'].close()
    config['LOG_FILE_HDL'].baseFilename = os.path.abspath(config['LOG_FILE'])

    config['DEBUG_LOG_HDL'].close()
    config['DEBUG_LOG_HDL'].baseFilename = os.path.abspath(config['DEBUG_LOG'])
  else:
    format_str = '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
    formatter = logging.Formatter(format_str)

    log = logging.getLogger()

    log.setLevel(logging.DEBUG)

    # add file mode="w" to overwrite
    config['LOG_FILE_HDL'] = logging.FileHandler(config['LOG_FILE'], mode='a')
    config['LOG_FILE_HDL'].setLevel(logging.INFO)
    config['LOG_FILE_HDL'].setFormatter(formatter)
    log.addHandler(config['LOG_FILE_HDL'])

    # the delay=1 should prevent the file from being opened until used.
    config['DEBUG_LOG_HDL'] = logging.FileHandler(config['DEBUG_LOG'], mode='a', delay=1)
    config['DEBUG_LOG_HDL'].setLevel(logging.DEBUG)
    config['DEBUG_LOG_HDL'].setFormatter(formatter)
    log.addHandler(config['DEBUG_LOG_HDL'])

    ch = logging.StreamHandler()
    ch.setLevel(logging.DEBUG)
    ch.setFormatter(formatter)
    log.addHandler(ch)
    config['LOGGING_SET'] = True

Answer 2

Indeed, logging.basicConfig does nothing if a handler has been set up already:

This function does nothing if the root logger already has handlers configured for it.

You'll need to replace the current handler on the root logger:

import logging

fileh = logging.FileHandler('/tmp/logfile', 'a')
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
fileh.setFormatter(formatter)

log = logging.getLogger()  # root logger
for hdlr in log.handlers[:]:  # remove all old handlers
    log.removeHandler(hdlr)
log.addHandler(fileh)      # set the new handler

See the Configuring Logging chapter in the Python Logging HOWTO.

Answer 3

I tried to implemented the suggestions on this page from @Martijn Pieters combined with @Arun Thundyill Saseendran. I'm too new to be allowed to comment so I have to post an adjusted answer. In the isinstance call, I had to use 'logging' instead of 'log' to get access to the types (log was an instance) and then the 'FileHander' should be 'FileHandler'. I'm using Python 3.6.

import logging

filehandler = logging.FileHandler('/tmp/logfile', 'a')
formatter = logging.Formatter('%(asctime)-15s::%(levelname)s::%(filename)s::%(funcName)s::%(lineno)d::%(message)s')
filehandler.setFormatter(formatter)
log = logging.getLogger()  # root logger - Good to get it only once.
for hdlr in log.handlers[:]:  # remove the existing file handlers
    if isinstance(hdlr,logging.FileHandler): #fixed two typos here
        log.removeHandler(hdlr)
log.addHandler(filehandler)      # set the new handler
# set the log level to INFO, DEBUG as the default is ERROR
logging.setLevel(log.DEBUG)      

Answer 4

The answer provided by @Martijn Pieters works good. However, the code snipper removes all handlers and placed only the file handler back. This will be troublesome if your application has handlers added by other modules.

Hence, the below snippet is designed in such a way to replace only the file handler.

The line if isinstance(hdlr,logging.FileHandler) is the key.

import logging

filehandler = logging.FileHandler('/tmp/logfile', 'a')
formatter = logging.Formatter('%(asctime)-15s::%(levelname)s::%(filename)s::%(funcName)s::%(lineno)d::%(message)s')
filehandler.setFormatter(formatter)
log = logging.getLogger()  # root logger - Good to get it only once.
for hdlr in log.handlers[:]:  # remove the existing file handlers
    if isinstance(hdlr,logging.FileHandler):
        log.removeHandler(hdlr)
log.addHandler(filehandler)      # set the new handler
# set the log level to INFO, DEBUG as the default is ERROR
log.setLevel(logging.DEBUG)