How to remove timestamps from celery pprint output?

问题

When running the celery worker then each line of the output of the pprint is always prefixed by the timestamp and also is being stripped. This makes it quite unreadable:

[2015-11-05 16:01:12,122: WARNING/Worker-2] {
[2015-11-05 16:01:12,122: WARNING/Worker-2] u'key1'
[2015-11-05 16:01:12,122: WARNING/Worker-2] :
[2015-11-05 16:01:12,122: WARNING/Worker-2] 'value1'
[2015-11-05 16:01:12,122: WARNING/Worker-2] ,
u'_id':
[2015-11-05 16:01:12,122: WARNING/Worker-2] ObjectId('55fff3b74322c53d18ae4687')
...

Is there a way how to tell celery not to format the output of pprint?

UPDATE:

the question was placed a bit wrong. The desired output would look something like this:

[2015-11-05 16:01:12,122: WARNING/Worker-2] 
{
    u'key1': 'value1',
    u'_id': ObjectId('55fff3b74322c53d18ae4687'),
    ...

回答1:

As @xbirkettx mentioned the output format is deffined in the CELERYD_LOG_FORMAT setting, which defaults to [%(asctime)s: %(levelname)s/%(processName)s] %(message)s.

So, in your settings:

CELERYD_LOG_FORMAT = '[%(levelname)s/%(processName)s] %(message)s'

There's also a special setting for a in-task logger, which defaults to:

CELERYD_TASK_LOG_FORMAT = [%(asctime)s: %(levelname)s/%(processName)s]
[%(task_name)s(%(task_id)s)] %(message)s

Remove the asctime key to get rid of the timestamp. Docs on CELERYD_TASK_LOG_FORMAT.

Update

From the docs:

You can also use print(), as anything written to standard out/-err will be redirected to the logging system (you can disable this, see CELERY_REDIRECT_STDOUTS).

So, rather than calling pprint.pprint, it is better to format a string with pprint.pformat and then log it.

@periodic_task(run_every=timedelta(seconds=10))
def pprint_dict2():
    import pprint
    values = {
        u'key1': 'value1',
        u'_id1': "ObjectId('55fff3b74322c53d18ae4687')",
        u'key2': 'value2',
        u'_id2': "ObjectId('55fff3b74322c53d18ae4687')",
        u'key3': 'value3',
        u'_id3': "ObjectId('55fff3b74322c53d18ae4687')",
        u'key4': 'value4',
        u'_id3': "ObjectId('55fff3b74322c53d18ae4687')",
    }
    s = pprint.pformat(values, width=1)
    print(s)  # or even better logger.info(...)

Output:

[WARNING/Beat] {u'_id1': "ObjectId('55fff3b74322c53d18ae4687')",
 u'_id2': "ObjectId('55fff3b74322c53d18ae4687')",
 u'_id3': "ObjectId('55fff3b74322c53d18ae4687')",
 u'key1': 'value1',
 u'key2': 'value2',
 u'key3': 'value3',
 u'key4': 'value4'}

回答2:

Try playing around with CELERYD_LOG_FORMAT

http://docs.celeryproject.org/en/latest/configuration.html

来源：https://stackoverflow.com/questions/33548095/how-to-remove-timestamps-from-celery-pprint-output

标签

python

celery

pprint