Celery - run different workers on one server

Question

I have 2 kind of tasks : Type1 - A few of high priority small tasks. Type2 - Lot of heavy tasks with lower priority.

Initially i had simple configuration with default ro

Answer 1

Another alternative is to give the worker process a unique name -- using the -n argument.

I have two Pyramid apps running on the same physical hardware, each with its own celery instance(within their own virtualenvs).

They both have Supervisor controlling both of them, both with a unique supervisord.conf file.

app1:

[program:celery]                                            
autorestart=true                                            
command=%(here)s/../bin/celery worker -n ${HOST}.app1--app=app1.queue -l debug
directory=%(here)s     

[2013-12-27 10:36:24,084: WARNING/MainProcess] celery@maz.local.app1 ready.

app2:

[program:celery]                                 
autorestart=true                                 
command=%(here)s/../bin/celery worker -n ${HOST}.app2 --app=app2.queue -l debug
directory=%(here)s                               

[2013-12-27 10:35:20,037: WARNING/MainProcess] celery@maz.local.app2 ready.