How to use python logging in multiple modules

Question

I was wondering what the standard set up is for performing logging from within a Python app.

I am using the Logging class, and I\'ve written my own logger class that

Answer 1

I don't know what you mean by the Logging class - there's no such class in Python's built-in logging. You don't really need wrappers: here's an example of how to do logging from arbitrary classes that you write:

import logging

# This class could be imported from a utility module
class LogMixin(object):
    @property
    def logger(self):
        name = '.'.join([__name__, self.__class__.__name__])
        return logging.getLogger(name)


# This class is just there to show that you can use a mixin like LogMixin
class Base(object):
    pass

# This could be in a module separate from B
class A(Base, LogMixin):
    def __init__(self):
        # Example of logging from a method in one of your classes
        self.logger.debug('Hello from A')

# This could be in a module separate from A
class B(Base, LogMixin):
    def __init__(self):
        # Another example of logging from a method in one of your classes
        self.logger.debug('Hello from B')

def main():
    # Do some work to exercise logging
    a = A()
    b = B()
    with open('myapp.log') as f:
        print('Log file contents:')
        print(f.read())

if __name__ == '__main__':
    # Configure only in your main program clause
    logging.basicConfig(level=logging.DEBUG,
                        filename='myapp.log', filemode='w',
                        format='%(name)s %(levelname)s %(message)s')
    main()

Generally it's not necessary to have loggers at class level: in Python, unlike say Java, the unit of program (de)composition is the module. However, nothing stops you from doing it, as I've shown above. The script, when run, displays:

Log file contents:
__main__.A DEBUG Hello from A
__main__.B DEBUG Hello from B

Note that code from both classes logged to the same file, myapp.log. This would have worked even with A and B in different modules.

Answer 2

Try using logging.getLogger() to get your logging object instance:

http://docs.python.org/3/library/logging.html#logging.getLogger

All calls to this function with a given name return the same logger instance. This means that logger instances never need to be passed between different parts of an application.

UPDATE:

The recommended way to do this is to use the getLogger() function and configure it (setting a handler, formatter, etc...):

# main.py
import logging
import lib


def main():
    logger = logging.getLogger('custom_logger')
    logger.setLevel(logging.INFO)
    logger.addHandler(logging.FileHandler('test.log'))
    logger.info('logged from main module')
    lib.log()

if __name__ == '__main__':
    main()

# lib.py
import logging


def log():
    logger = logging.getLogger('custom_logger')
    logger.info('logged from lib module')

If you really need to extend the logger class take a look at logging.setLoggerClass(klass)

UPDATE 2:

Example on how to add a custom logging level without changing the Logging class:

# main.py
import logging
import lib


# Extend Logger class
CUSTOM_LEVEL_NUM = 9
logging.addLevelName(CUSTOM_LEVEL_NUM, 'CUSTOM')
def custom(self, msg, *args, **kwargs):
    self._log(CUSTOM_LEVEL_NUM, msg, args, **kwargs)
logging.Logger.custom = custom

# Do global logger instance setup
logger = logging.getLogger('custom_logger')
logger.setLevel(logging.INFO)
logger.addHandler(logging.FileHandler('test.log'))


def main():
    logger = logging.getLogger('custom_logger')
    logger.custom('logged from main module')
    lib.log()

if __name__ == '__main__':
    main()

Note that adding custom level is not recommended: http://docs.python.org/2/howto/logging.html#custom-levels

Defining a custom handler and maybe using more than one logger may do the trick for your other requirement: optional output to stderr.