Run a celery worker in the background

Question

I am running a celery worker like this:

celery worker --app=portalmq --logfile=/tmp/portalmq.log --loglevel=INFO -E --pidfile=/tmp/portalmq.pid

Answer 1

I have faced the same problem as a lazy solution is to use & at the end of the command. For example

celery worker -A <app>.celery --loglevel=info &  

Answer 2

supervisor is really simple and requires really little work to get it setup up, same applies for to celery in combination with supervisor.

It should not take more than 10 minutes to setup it up :)

1. install supervisor with apt-get

2. create /etc/supervisor/conf.d/celery.conf config file

3. paste somethis in the celery.conf file

   [program:celery]
   directory = /my_project/
   command = /usr/bin/python manage.py celery worker
   

4. plus (if you need) some optional and useful stuff (with dummy values)

   user = celery_user
   group = celery_group
   stdout_logfile = /var/log/celeryd.log
   stderr_logfile = /var/log/celeryd.err
   autostart = true
   environment=PATH="/some/path/",FOO="bar"
   

5. restart supervisor (or do supervisorctl reread; supervisorctl add celery)

after that you get the nice ctl commands to manage the celery process:

supervisorctl start/restart/stop celery

supervisorctl tail [-f] celery [stderr]

Answer 3

celery worker -A app.celery --loglevel=info --detach