Celery Starting a Task when Other Tasks have Completed

问题

I have 3 tasks in Celery..

celery_app.send_task('tasks.read_cake_recipes')
celery_app.send_task('tasks.buy_ingredients')

celery_app.send_task('tasks.make_cake')

Both read_cake_recipes and buy_ingredients don't have any dependancies, however before the task make_cake can be run both read_cake_recipes and buy_ingredients need to have finished.

make_cake can be run at ANYTIME after the first two have started. But make_cake has no idea if the other tasks have completed. So if read_cake_recipes or buy_ingredients takes too long, then make_cake fails miserably.

Chaining tasks does not seem to work here because make_cake has more than one dependancies.

How can I still start the task make_cake, but have it then wait/pending etc until the other two tasks have completed first?

My saving grace is that read_cake_recipes and buy_ingredients save results to the DB, if make_cake somehow knew which ingredients or recipes to look for it could check this, maybe?

回答1:

Totally guessing at your underlying architecture, but here goes..

class Cake(models.Model):
    recipes_read = models.BooleanField(default=False)
    ingredients_purchased = models.BooleanField(default=False)
    batter_prepared = models.BooleanField(default=False)

    def save(self, *args, **kwargs):
        if self.recipes_read and self.ingredients_purchased:
            self.batter_prepared = True
        super(Cake, self).save(*args, **kwargs)


@task
read_cake_recipes():
    for cake in Cake.objects.all():
        # Read some shit!
        cake.recipes_read = True
        cake.save()

@task
buy_cake_ingredients():
    for cake in Cake.objects.all():
        # Buy some shit!
        cake.ingredients_purchased = True
        cake.save()

@task
make_cake():
    for cake in Cake.objects.filter(batter_prepared=True):
        # Make that shit!

来源：https://stackoverflow.com/questions/25576890/celery-starting-a-task-when-other-tasks-have-completed

标签

python

django

celery