celery .env variables not used in settings.py

问题

I am stuck on using config vars from my .env file inside the settings.py for my celery.py.

when i hardcode CELERY_BROKER_URL = 'redis://localhost', everything works, however, when i use CELERY_BROKER_URL= os.environ.get('REDIS_URL'), the REDIS_URL is not taken over and I get an error.

celery.py:

from __future__ import absolute_import, unicode_literals
import os
from celery import Celery

# set the default Django settings module for the 'celery' program.
os.environ['DJANGO_SETTINGS_MODULE'] = 'xlink.settings'

app = Celery('xlink')

# Using a string here means the worker doesn't have to serialize
# the configuration object to child processes.
# - namespace='CELERY' means all celery-related configuration keys
#   should have a `CELERY_` prefix.
app.config_from_object('django.conf:settings', namespace='CELERY')

# Load task modules from all registered Django app configs.
app.autodiscover_tasks()


@app.task(bind=True)
def debug_task(self):
    print('Request: {0!r}'.format(self.request))

.env:

REDIS_URL = 'redis://localhost'
REDIS_PASSWORD = 'yow'
REDIS_HOST = 'localhost'

NOT WORKING settings.py version 1:

CELERY_BROKER_URL= os.environ.get('REDIS_URL')
CELERY_RESULT_BACKEND= os.environ.get('REDIS_URL')

error version 1 (trying to take over the default redis set-up as noting is given):

[2017-08-26 10:57:09,253: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@127.0.0.1:5672//: [Errno 61] Connection refused.
Trying again in 2.00 seconds...

NOT WORKING settings.py version 2:

CELERY_BROKER_URL = 'redis://:{}@{}:6379/0'.format(
    os.environ.get('REDIS_PASSWORD'),
    os.environ.get('REDIS_HOST'))

error version 2 (clearly not taking over .env variables):

[2017-08-26 11:11:03,419: ERROR/MainProcess] consumer: Cannot connect to redis://:**@none:6379/0: Error 8 connecting to none:6379. nodename nor servname provided, or not known..
Trying again in 2.00 seconds...

WORKING settings.py:

CELERY_BROKER_URL= 'redis://localhost'
CELERY_RESULT_BACKEND= 'redis://localhost'

tasks.py:

import celery
from celery import shared_task

@shared_task
def add(x, y):
    return x + y

回答1:

The big mistake I made was concerning running the celery worker separate from the commandline with this code:

celery -A xlink worker -l info

adding this line of code to the procfile and running with the heroku command solved this issue.

profile:

web: gunicorn xlink.wsgi --threads 4 --log-level debug
worker: celery -A xlink worker -l info

command to get it running:

heroku local worker

If you want to run both the worker and the web app in one terminal window, just do heroku local and both start simultaneously in one terminal window.

thanks for the hints @Daniel Roseman

来源：https://stackoverflow.com/questions/45894658/celery-env-variables-not-used-in-settings-py

标签

python

django

celery