Reporting yielded results of long-running Celery task

Question

Problem

I\'ve segmented a long-running task into logical subtasks, so I can report the results of each subtask as it completes. However, I\'m trying to report the

Answer 1

A couple options to consider:

1 -- task groups. If you can enumerate all the sub tasks from the time of invocation, you can apply the group as a whole -- that returns a TaskSetResult object you can use to monitor the results of the group as a whole, or of individual tasks in the group -- query this as-needed when you need to check status.

2 -- callbacks. If you can't enumerate all sub tasks (or even if you can!) you can define a web hook / callback that's the last step in the task -- called when the rest of the task completes. The hook would be against a URI in your app that ingests the result and makes it available via DB or app-internal API.

Some combination of these could solve your challenge.