Tensorflow logging messages do not appear

问题

I use tensorflow 1.2.0 installed with pip install.

When I run samples that include

import logging
tf.logging.set_verbosity(tf.logging.INFO)

the logging messages of the form

logging.info('TEST')

do not appear in the terminal output, even with the flag --tostderr.

According to this answer I also tried

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '0'

but still the problem persists. Any ideas?

回答1:

TF Logging Basics:

So there is a lot of confusion around tensorflow logging, and it is really not well documented. I landed here a few times in my searches, so it seems to be a good place to post an answer.

After some research and experimentation with Ubuntu and Windows (more than I had planned), this is what I got:

There are two flags, similarly named, but with somewhat different semantics:

TF_CPP_MIN_LOG_LEVEL - which has 3 or 4 basic levels - low numbers = more messages.
- 0 outputs Information, Warning, Error, and Fatals (default)
- 1 outputs Warning, and above
- 2 outputs Errors and above.
- etc... I didn't check edge cases
TF_CPP_MIN_VLOG_LEVEL - which causes very very many extra Information errors - really for debugging only - low numbers = less messages.
- 3 Outputs lots and lots of stuff
- 2 Outputs less
- 1 Outputs even less
- 0 Outputs nothing extra (default)

Additional Notes:

Since all the VLOG messages are Informational, then LOG needs to be set at 0 for you to see them. Fortunately that is the default.
These errors go to the standard error so you can redirect them with something like:
- python tf-program.py &>mylog.log
These are supposed to be picked up by the os module so you should be able to set them in the environment
Without the VLOG and with no GPU there are not that many information messages, so you can think logging is not working when it really is.

Windows:

Except python's os module did not pick them up under Windows. Python never loved Windows...
- this code sequence works for me in Windows (and would surely work in Linux):
  - import os
  - os.environ['TF_CPP_MIN_LOG_LEVEL'] = '0'
  - os.environ['TF_CPP_MIN_VLOG_LEVEL'] = '3'
  - import tensorflow as tf

Linux:

Under Linux (bash) you can specify these conveniently on the command line, so with something like:
- TF_CPP_MIN_VLOG_LEVEL=3 python tf-program.py

FWIW, I tested on TensorFlow 1.7 with this tutorial:

https://github.com/tensorflow/models/tree/master/tutorials/image/mnist

And this is what it looks like:

回答2:

There are really two logging systems in tensorflow: one in the C++ core (specifically tensorflow/core/platform/default/logging.{h,cc}) and the other in the Python bindings. The two systems are independent.

The one in the Python binding plays nicely with the standard Python logging module. The C++ logger is indepedently controlled by the TF_CPP_MIN_LOG_LEVEL environment variables mentioned in previous answers.

The following Python (that I locally called logtest.py) demonstrates the two systems' independence.

"""
Demonstrate independence of TensorFlow's two logging subsystems.
"""

import argparse
import tensorflow as tf
import logging

_logger = logging.getLogger( "tensorflow" )

parser = argparse.ArgumentParser( description="Demo TensorFlow logging" )

parser.add_argument("-v","--verbosity",
    default="",
    choices=['DEBUG','INFO','WARNING','ERROR','CRITICAL'],
    help="One of {DEBUG,INFO,WARNING,ERROR,CRITICAL}" )

args = parser.parse_args()

print( "Initial Python effective log level:", _logger.getEffectiveLevel() )

# If user provided an explicit Python level, set it.

if args.verbosity:
    _logger.setLevel( args.verbosity   )
    print( " ...new Python effective log level:", _logger.getEffectiveLevel() ) # ...and confirm the change.

_logger.debug(    "   DEBUG messages are emitted" )
_logger.info(     "    INFO messages are emitted" )
_logger.warn(     " WARNING messages are emitted" )
_logger.error(    "   ERROR messages are emitted" )
_logger.critical( "CRITICAL messages are emitted" )

with tf.Session() as s:
    pass # ...just to trigger TensorFlow into action to generate logging.

Running...

TF_CPP_MIN_LOG_LEVEL=0 python3 logtest.py -v CRITICAL

...shows that Python can't silence the core logging system, and

TF_CPP_MIN_LOG_LEVEL=5 python3 logtest.py -v DEBUG

...shows that the core system can't silence Python.

回答3:

What I usually do to control the TensorFlow logging is to have this piece of code before any TensorFlow import

import os
import logging
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '0'
logging.getLogger("tensorflow").setLevel(logging.WARNING)

import tensorflow as tf

I'd be happy to hear about any better solution.

回答4:

I tried to set TF_CPP_MIN_LOG_LEVEL but still not work. after check this thread https://github.com/tensorflow/tensorflow/issues/1258

as it said

It's TF_CPP_MIN_VLOG_LEVEL, not TF_CPP_MIN_LOG_LEVEL Also, note that if TF_CPP_MIN_LOG_LEVEL is set, then TF_CPP_MIN_VLOG_LEVEL values are ignored

then I unset TF_CPP_MIN_LOG_LEVEL and set TF_CPP_MIN_VLOG_LEVEL again, it works.

the two macro makes me confused hope it helps.

来源：https://stackoverflow.com/questions/44853059/tensorflow-logging-messages-do-not-appear

标签

python

logging

tensorflow

error-logging