Celery: list all tasks, scheduled, active *and* finished

Question

Update for the bounty

I'd like a solution that does not involve a monitoring thread, if possible.

---

I know I can view scheduled and active tasks using the Inspect class of my apps Control.

i = myapp.control.inspect()

currently_running = i.active()
scheduled = i.scheduled()

But I could not find any function to show already finished tasks. I know that this information mus be at least temporarily accessible, because I can look up a finished task by its task_id:

>>> r = my task.AsyncResult(task_id=' ... ')
>>> r.state
u'SUCCESS'

How can I get a complete list of scheduled, active and finished tasks? Or possibly a list of all tasks at once?

Answer 1

Celery Flower shows tasks (active, finished, reserved, etc) in real time. It enables to filter tasks by time, workers and types.

https://github.com/mher/flower

Answer 2

One option not requiring a monitoring thread is a Celery on_success handler (using bootsteps feature in 3.1+) - this would need to write relevant info to your own datastore.

• You need to create a custom task class to do this. This on_failure example gives an idea.

Possibly better option, needing less code, is to use a task_success signal in a similar way, recording the info you need later.

The Flower option is probably simpler, as you are querying info already maintained by Flower when tasks complete - see this answer.