Celery worker and command line args

问题

I am refactoring my code to use celery worker.

Before I used to use argparse to pass command line args.

e.g.

if __name__ == "__main__":
     parser = argparse.ArgumentParser(description='Node')
     parser.add_argument('--environment', action="store", default='local', help="env e.g. production of development")
     environment = arg_options.environment

But now I get this error.

celery -A tasks worker --loglevel=info --environment local
celery: error: no such option: --environment

How can I add?

I don't want to use environment variable if I don't have to.

e.g export environment=development

回答1:

The Celery worker does not execute your __main__.

If you want to add additional command-line options you can use app.user_options, but note that it uses the optparse module, not argparse.

See this section in the docs for more:

http://docs.celeryproject.org/en/latest/userguide/extending.html#preload-options

回答2:

By looking at the source code, I've figured out a way to handle this.

On celery.py add:

from celery import bootsteps
from celery.bin import Option

....
app.user_options['worker'].add(
    Option('--server', dest='api_server', default='127.0.0.1',
           help='API server to use.')
)

app.conf['API_SERVER'] = '127.0.0.1'

class ConfigBootstep(bootsteps.Step):
    def __init__(self, worker, api_server=None, **options):
        app.conf['API_SERVER'] = api_server

app.steps['worker'].add(ConfigBootstep)

Then in the file holding your tasks add this:

from celery import current_app
...
@shared_task()
def roboscope():
    API_SERVER = current_app.conf.get('API_SERVER', '127.0.0.1')

I tried making API_SERVER a module global variable by resolving it when the module is imported but it doesn't work as it's too early. As my task is quite intensive it doesn't harm to run this many times.

来源：https://stackoverflow.com/questions/21365101/celery-worker-and-command-line-args

标签

python

celery