问题
A related (albeit not identical) question appears here: Interact with celery ongoing task
It's easy to start a task and get its unique ID:
async_result = my_task.delay()
task_id = async_result.task_id
It's easy to broadcast a message that will reach a custom command in the worker:
my_celery_app.control.broadcast('custom_command', arguments= {'id': task_id})
The problem arises that the worker is started in the form of a small process tree formed of one supervisor and a number of children. The supervisor receives messages and acts on them, delegating task execution to children but custom commands are run in the supervisor.
I can find no clear documentation or means by which a custom command running in the supervisor can send a message to specific child that is running the task identified, or even broadcast a message down to all children so the the child running the task identified can act on it.
The supervisor delegates work to children and has some line of communication open, but I cna't find any documented means of exploiting it.
It's possible to write a custom control and task pair that communicate with each other independently of the Celery relationship using either kombu (as Celery does) or some other means (python 3.8 enticingly opens the possibility of shared memory objects).
But it would see far more appropriate and logical if Celery had a simple API through which a custom control can pass a message to a running task (either to the child process running it, or broadcast to all children with a task_id as a parameter enabling the right one to act on it).
It's conceivable that:
- Such an API exists in Celery and I've not found it
- There's a backdoor means of accessing Celery functions to send such a message
- No such facility exists in Celery
It's a generic problem, and it would be useful given the enormous utility an popularity of Celery, if we had some means of interacting with a running task of a given task_id, to send it messages.
The result backend provides the means of receiving messages from running tasks. What seems unclear or missing is how to send messages to running tasks.
An example:
In its default configuration (i.e. using the preform pool configuration) if I run celery -A myapp worker it produces processes like this pstree output:
[celeryd: celer(19955)─┬─[celeryd: celer(19957)
├─[celeryd: celer(19958)
├─[celeryd: celer(19959)
├─[celeryd: celer(19960)
├─[celeryd: celer(19961)
├─[celeryd: celer(19962)
├─[celeryd: celer(19963)
└─[celeryd: celer(19964)
That is the worker (supervisor) starts 8 children (the preforked pool) and it is the supervisor that receives messages (via RabbitMQ) ... notable to start a task. It then (I assume) sends a message (using kombu it seems which presumably also uses RabbitMQ) to one of the children in the pool to run the task.
The challenge now, is for the client, that started the task, which knows the task ID, to interact with the actually running function in the child process.
We can get half way there with a custom control command:
http://docs.celeryproject.org/en/latest/userguide/workers.html#writing-your-own-remote-control-commands
But this command runs in the worker (supervisor) and so to communicate with the running function (send it a request) we need a mechanism to do so. And the question is, does Celery provide one I've not found, either documented, or not (internal) and if not has anyone implemented another means?
来源：https://stackoverflow.com/questions/59796397/celery-interact-communicate-with-a-running-task