How to setup celery worker to log all task function calls to one file

那年仲夏 提交于 2019-12-11 15:20:56

问题


I have Django application with such logging configuration.

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'default': {
            'format': '%(asctime)s [%(levelname)s] %(filename)s:%(lineno)s: %(message)s'
        },
    },
    'handlers': {
        'cron': {
            'class': 'logging.FileHandler',
            'filename': 'cron.log',
            'formatter': 'default',
        },
        'admin': {
            'class': 'logging.FileHandler',
            'filename': 'admin.log',
            'formatter': 'default',
        },
        'app': {
            'class': 'logging.FileHandler',
            'filename': 'app.log',
            'formatter': 'default',
        },
        'core': {
            'class': 'logging.FileHandler',
            'filename': 'app.log',
            'formatter': 'default',
        },
        'metrics': {
            'class': 'logging.FileHandler',
            'filename': 'metrics.log',
            'formatter': 'default',
        },
        'pixel': {
            'class': 'logging.FileHandler',
            'filename': 'pixel.log',
            'formatter': 'default',
        },
    },
    'loggers': {
        'cron_api': {
            'handlers': ['cron'],
            'level': 'DEBUG',
        },
        'main': {
            'handlers': ['app'],
            'level': 'INFO',
        },
        'admin': {
            'handlers': ['admin'],
            'level': 'INFO',
        },
        'spa_api': {
            'handlers': ['app'],
            'level': 'INFO',
        },
        'metrics': {
            'handlers': ['metrics'],
            'level': 'INFO',
        },
        'pixel': {
            'handlers': ['pixel'],
            'level': 'INFO',
        },
    },
}

Also I have a celery worker. I want to log every activity that happens inside the task to celery_worker.log, even if task calls a function from main or cron_api application.

For example: If main.celery_tasks.task1.task1 calls main.service.google.get_campaign, I want all logs from get_campaign to be in worker.log. Also I want no logs related to this task to be in app.log.

How to achieve this? P.S.: Also I'd like to have a scalable solution. If number of worker increases - I'd like each worker to write all logs related to task call to it's file.

来源:https://stackoverflow.com/questions/57296160/how-to-setup-celery-worker-to-log-all-task-function-calls-to-one-file

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!