JSON formatted logging with Flask and gunicorn

Question

I am trying to do something very similar to what's explained here: https://sebest.github.io/post/protips-using-gunicorn-inside-a-docker-image/

I want to get my Flask app + gunicorn both outputting JSON formatted logging. I've got this working for the Flask app, but I can't seem to get it working with gunicorn.

Here's my current output:

 * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
 * Restarting with stat
 * Debugger is active!
 * Debugger pin code: 317-187-130
192.168.99.1 - - [12/Jan/2016 20:09:29] "GET /nothing HTTP/1.1" 404 -
{"asctime": "2016-01-12 20:20:25,541", "levelname": "WARNING", "module": "app", "funcName": "err", "lineno": 17, "message": "will assert false..."}
192.168.99.1 - - [12/Jan/2016 20:20:25] "GET /err HTTP/1.1" 500 -

The line that begins {"asctime": is the output of the code app.logger.warning('will assert false...') which is being correctly logged as JSON. Yay. The lines that begin with 192.168.99.1 are output from my gunicorn WSGI server and, frustratingly, they are not JSON formatted.

Here's the command I am using to start gunicorn:

gunicorn --log-config gunicorn_logging.conf -c gunicorn_config.py api.app:app

where the gunicorn_logging.conf file contains:

[loggers]
keys=root, gunicorn.error

[handlers]
keys=console

[formatters]
keys=json

[logger_root]
level=INFO
handlers=console

[logger_gunicorn.error]
level=ERROR
handlers=console
propagate=0
qualname=gunicorn.error

[handler_console]
class=StreamHandler
formatter=json
args=(sys.stdout, )

[formatter_json]
class=jsonlogging.JSONFormatter

and the file gunicorn_config.py contains:

import os
import multiprocessing

addr = os.environ.get('HTTP_ADDR', '0.0.0.0')
port = os.environ.get('HTTP_PORT', '5000')
loglevel = os.environ.get('LOG_LEVEL', 'info')

bind = '{0}:{1}'.format(addr, port)
workers = multiprocessing.cpu_count() * 5 + 1
worker_class = 'gevent'
timeout = 0

Here's the output of pip freeze:

aniso8601==1.1.0
coverage==4.0.3
flake8==2.5.1
Flask==0.10.1
Flask-MySQLdb==0.2.0
Flask-RESTful==0.3.5
Flask-Script==2.0.5
gevent==1.1rc3
greenlet==0.4.9
gunicorn==19.4.5
itsdangerous==0.24
Jinja2==2.8
json-logging-py==0.2
MarkupSafe==0.23
marshmallow==2.4.2
mccabe==0.3.1
mysqlclient==1.3.7
nose==1.3.7
pep8==1.5.7
pyflakes==1.0.0
python-dateutil==2.4.2
python-json-logger==0.1.4
pytz==2015.7
six==1.10.0
SQLAlchemy==1.0.11
Werkzeug==0.11.3

Answer 1

to get the access log as well your config file should look something like this:

[loggers]
keys=root, gunicorn.error, gunicorn.access
...
[logger_gunicorn.access]
level=(DEBUG|INFO|ERROR)
handlers=console
propagate=0
qualname=gunicorn.access

The accesses are logged in the INFO log level, so if you want to show them make sure you have the log level within the "logger_gunicorn.access" tag set to at least INFO