celery

I'm using Celery (3.0.15) with Redis as a broker. Is there a straightforward way to query the number of tasks with a given name that exist in a Celery queue? And, as a followup, is there a way to cancel all tasks with a given name that exist in a Celery queue? I've been through the Monitoring and Management Guide and don't see a solution there. # Retrieve tasks # Reference: http://docs.celeryproject.org/en/latest/reference/celery.events.state.html query = celery.events.state.tasks_by_type(your_task_name) # Kill tasks # Reference: http://docs.celeryproject.org/en/latest/userguide/workers.html

While issuing a new build to update code in workers how do I restart celery workers gracefully? Edit: What I intend to do is to something like this. Worker is running, probably uploading a 100 MB file to S3 A new build comes Worker code has changes Build script fires signal to the Worker(s) Starts new workers with the new code Worker(s) who got the signal after finishing the existing job exit. The new recommended method of restarting a worker is documented in here http://docs.celeryproject.org/en/latest/userguide/workers.html#restarting-the-worker $ celery multi start 1 -A proj -l info -c4 -

The Celery documentation mentions testing Celery within Django but doesn't explain how to test a Celery task if you are not using Django. How do you do this? It is possible to test tasks synchronously using any unittest lib out there. I normaly do 2 different test sessions when working with celery tasks. The first one (as I'm suggesting bellow) is completely synchronous and should be the one that makes sure the algorithm does what it should do. The second session uses the whole system (including the broker) and makes sure I'm not having serialization issues or any other distribution,

Things changed too much in Django, so I can't use 3.1. I need some help. I read about make a task in django , and read Periodic Tasks document. But I don't know how make periodic tasks in django. I think this becuase of my low level English.. In the older version of Celery, I imported djcelery & crontab and set CELERYBEAT_SCHEDULE in settings.py , and excuted by manage.py . But it seems that I cannot execute celery deamon by that way anymore. Than where I should put CELERYBEAT_SCHEDULE? In django example in docs, they set os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'proj.settings') in proj

How can I delete all pending tasks without knowing the task_id for each task? Philip Southam From the docs : $ celery -A proj purge or from proj.celery import app app.control.purge() (EDIT: Updated with current method.) ToonAlfrink For celery 3.0+: $ celery purge To purge a specific queue: $ celery -Q queue_name purge smido For Celery 2.x and 3.x: When using worker with -Q parameter to define queues, for example celery worker -Q queue1,queue2,queue3 then celery purge will not work, because you cannot pass the queue params to it. It will only delete the default queue. The solution is to start

I am new to message brokers like RabbitMQ which we can use to create tasks / message queues for a scheduling system like Celery . Now, here is the question: I can create a table in PostgreSQL which can be appended with new tasks and consumed by the consumer program like Celery. Why on earth would I want to setup a whole new tech for this like RabbitMQ? Now, I believe scaling cannot be the answer since our database like PostgreSQL can work in a distributed environment. I googled for what problems does the database poses for the particular problem, and I found: polling keeps the database busy