Keras interfering with python logging

问题

I want to log some python code which loads in a trained keras model. For some reason, (python) logging doesn't not work if the keras load_model is imported. But (python) logging works fine if I don't import keras.

This works:

import logging

LOG_FORMAT = "%(levelname)s %(asctime)s - %(message)s"
logging.basicConfig(filename="logs/my_logs.log",
                    level=logging.DEBUG,
                    format=LOG_FORMAT)
logger = logging.getLogger()

def do_stuff():
    logging.info("About to do stuff")
    ... stuff gets done here...

This doesn't work

import logging
from keras.models import load_model

my_model = load_model("fetch_preprocess/lstm_model.h5")

LOG_FORMAT = "%(levelname)s %(asctime)s - %(message)s"
logging.basicConfig(filename="logs/my_logs.log",
                    level=logging.DEBUG,
                    format=LOG_FORMAT)
logger = logging.getLogger()

def do_stuff():
    logging.info("About to do stuff")
    pred = my_model.predict(data)
    ... stuff gets done here...

By "doesn't work", I mean the logging module doesn't

create a new log file
write to that file anytime logging is called

but no error's are ever thrown. So I find this strange.

I believe Keras / tensorflow is interfering with the logging (as the only change I make to the code is to exclude Keras and then logging works fine).

Is there some way to suppress whatever keras is doing in the background so I can use the python logging?

回答1:

This issue is because of the bug in the version 0.7.0 of the package, abseil and it has been fixed in its 0.8.0 version, as mentioned in this Github link.

The solution to this problem is to upgrade the version of the package abseil to greater than or equal to 0.8.

The sample code that

creates a new log file and
writes to that file anytime logging is called

is shown below:

!pip install --upgrade absl-py

import logging
from keras.models import load_model

my_model = load_model("fetch_preprocess/lstm_model.h5")

LOG_FORMAT = "%(levelname)s %(asctime)s - %(message)s"
logging.basicConfig(filename="logs/my_logs.log",
                    level=logging.DEBUG,
                    format=LOG_FORMAT)
logger = logging.getLogger()

def do_stuff():
    logging.info("About to do stuff")
    pred = my_model.predict(data)
    ... stuff gets done here...

do_stuff()

` Hope this helps. Happy Learning!

回答2:

As you mentioned, I also think that Keras is simply replacing your logger with its own. You can, however, do the following:

import logging as my_logging
from keras.models import load_model

my_model = load_model("fetch_preprocess/lstm_model.h5")

LOG_FORMAT = "%(levelname)s %(asctime)s - %(message)s"
my_logging.basicConfig(filename="logs/my_logs.log",
                    level=my_logging.DEBUG,
                    format=LOG_FORMAT)
my_logger = my_logging.getLogger()

来源：https://stackoverflow.com/questions/57836126/keras-interfering-with-python-logging

标签

python

tensorflow

logging

keras