Efficient recurring tasks in celery?

时光毁灭记忆、已成空白 提交于 2019-12-11 14:03:40

问题


I have ~250,000 recurring tasks each day; about a fifth of which might be updated with different scheduled datetimes each day.

Can this be done efficiently in Celery? - I am worried about this from celery's beat.py:

    def tick(self):
        """Run a tick, that is one iteration of the scheduler.

           Executes all due tasks.

       """
        remaining_times = []
        try:
            for entry in values(self.schedule):
                next_time_to_run = self.maybe_due(entry, self.publisher)
                if next_time_to_run:
                    remaining_times.append(next_time_to_run)
        except RuntimeError:
            pass

        return min(remaining_times + [self.max_interval])

来源:https://stackoverflow.com/questions/21274391/efficient-recurring-tasks-in-celery

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!