Redirect Python 'print' output to Logger

Question

I have a Python script that makes use of \'Print\' for printing to stdout. I\'ve recently added logging via Python Logger and would like to make it so these print statement

Answer 1

Of course, you can both print to the standard output and append to a log file, like this:

# Uncomment the line below for python 2.x
#from __future__ import print_function

import logging

logging.basicConfig(level=logging.INFO, format='%(message)s')
logger = logging.getLogger()
logger.addHandler(logging.FileHandler('test.log', 'a'))
print = logger.info

print('yo!')

Answer 2

Once your defined your logger, use this to make print redirect to logger even with mutiple parameters of print.

print = lambda *tup : logger.info(str(" ".join([str(x) for x in tup]))) 

Answer 3

One more method is to wrap the logger in an object that translates calls to write to the logger's log method.

Ferry Boender does just this, provided under the GPL license in a post on his website. The code below is based on this but solves two issues with the original:

1. The class doesn't implement the flush method which is called when the program exits.
2. The class doesn't buffer the writes on newline as io.TextIOWrapper objects are supposed to which results in newlines at odd points.

import logging
import sys


class StreamToLogger(object):
    """
    Fake file-like stream object that redirects writes to a logger instance.
    """
    def __init__(self, logger, log_level=logging.INFO):
        self.logger = logger
        self.log_level = log_level
        self.linebuf = ''

    def write(self, buf):
        temp_linebuf = self.linebuf + buf
        self.linebuf = ''
        for line in temp_linebuf.splitlines(True):
            # From the io.TextIOWrapper docs:
            #   On output, if newline is None, any '\n' characters written
            #   are translated to the system default line separator.
            # By default sys.stdout.write() expects '\n' newlines and then
            # translates them so this is still cross platform.
            if line[-1] == '\n':
                self.logger.log(self.log_level, line.rstrip())
            else:
                self.linebuf += line

    def flush(self):
        if self.linebuf != '':
            self.logger.log(self.log_level, self.linebuf.rstrip())
        self.linebuf = ''


logging.basicConfig(
    level=logging.DEBUG,
    format='%(asctime)s:%(levelname)s:%(name)s:%(message)s',
    filename="out.log",
    filemode='a'
)

stdout_logger = logging.getLogger('STDOUT')
sl = StreamToLogger(stdout_logger, logging.INFO)
sys.stdout = sl

stderr_logger = logging.getLogger('STDERR')
sl = StreamToLogger(stderr_logger, logging.ERROR)
sys.stderr = sl

This allows you to easily route all output to a logger of your choice. If needed, you can save sys.stdout and/or sys.stderr as mentioned by others in this thread before replacing it if you need to restore it later.

Answer 4

Below snipped works perfectly inside my PySpark code. If someone need in case for understanding -->

import os
import sys
import logging
import logging.handlers

log = logging.getLogger(__name_)

handler = logging.FileHandler("spam.log")
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
handler.setFormatter(formatter)
log.addHandler(handler)
sys.stderr.write = log.error 
sys.stdout.write = log.info 

(will log every error in "spam.log" in the same directory, nothing will be on console/stdout)

(will log every info in "spam.log" in the same directory,nothing will be on console/stdout)

to print output error/info in both file as well as in console remove above two line.

Happy Coding Cheers!!!

Answer 5

You have two options:

1. Open a logfile and replace sys.stdout with it, not a function:

   log = open("myprog.log", "a")
   sys.stdout = log
   
   >>> print("Hello")
   >>> # nothing is printed because it goes to the log file instead.
   

2. Replace print with your log function:

   # If you're using python 2.x, uncomment the next line
   #from __future__ import print_function
   print = log.info
   
   >>> print("Hello!")
   >>> # nothing is printed because log.info is called instead of print
   

Answer 6

A much simpler option,

import logging, sys
logging.basicConfig(filename='path/to/logfile', level=logging.DEBUG)
logger = logging.getLogger()
sys.stderr.write = logger.error
sys.stdout.write = logger.info