Do subtasks inherit the queue of their parent task?

只愿长相守 提交于 2019-12-11 10:07:29

问题


When creating a subtask (i.e. chord, chain, group) with Celery, and you have a multiple queues (i.e. high priority, low priority), does the subtask inherit the routing parameters of the task who created it?


回答1:


Answering my own question having actually read the source...

Short answer

No.

Long answer

Tasks instantiated with mytask.s() and mytask.si() call celery.app.Task.subtask() (called signature() in master), which does not set any routing information. Compare this to retry() which calls subtask_from_request(), which sets the queue from request.delivery_info.

My Solution

Subclass Task to add the queue to Task.subtask().

class Task(CeleryTask):
    """
    Override the Celery Task baseclass to send subtasks to the same
    queue as the main task.
    """

    def subtask(self, args=None, kwargs=None, options=None,
                *starargs, **starkwargs):

        kwargs = kwargs or {}
        options = options or {}

        # override the queue if not passed in as an option
        if set(('queue', 'exchange', 'routing_key')) & set(options.keys()):
            options.update(self.request.delivery_info)

        return super().subtask(args, kwargs, options,
                               *starargs, **starkwargs)

Use by passing base=Task to @task/@shared_task.




回答2:


In a word: No. The task will use the routing as documented here



来源:https://stackoverflow.com/questions/34123455/do-subtasks-inherit-the-queue-of-their-parent-task

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!