Celery Storing unrecoverable task failures for later resubmission

问题

I'm using the djkombu transport for my local development, but I will probably be using amqp (rabbit) in production.

I'd like to be able to iterate over failures of a particular type and resubmit. This would be in the case of something failing on a server or some edge case bug triggered by some new variation in data.

So I could be resubmitting jobs up to 12 hours later after some bug is fixed or a third party site is back up.

My question is: Is there a way to access old failed jobs via the result backend and simply resubmit them with the same params etc?

回答1:

You can probably access old jobs using:

CELERY_RESULT_BACKEND = "database"

and in your code:

from djcelery.models import TaskMeta
task = TaskMeta.objects.filter(task_id='af3185c9-4174-4bca-0101-860ce6621234')[0]

but I'm not sure you can find the arguments that the task is being started with ... Maybe something with TaskState...

I've never used it this way. But you might want to consider the task.retry feature? An example from celery docs:

@task()
def task(*args):
    try:
        some_work()
    except SomeException, exc:
        # Retry in 24 hours.
        raise task.retry(*args, countdown=60 * 60 * 24, exc=exc)

回答2:

From IRC

<asksol> dpn`: task args and kwargs are not stored with the result

<asksol> dpn`: but you can create your own model and store it there (for example using the task_sent signal)

<asksol> we don't store anything when the task is sent, only send a message. but it's very easy to do yourself

This was what I was expecting, but hoped to avoid.

At least I have an answer now :)

来源：https://stackoverflow.com/questions/11600792/celery-storing-unrecoverable-task-failures-for-later-resubmission

标签

django

celery

django-celery