Find out whether celery task exists

问题

Is it possible to find out whether a task with a certain task id exists? When I try to get the status, I will always get pending.

>>> AsyncResult('...').status
'PENDING'

I want to know whether a given task id is a real celery task id and not a random string. I want different results depending on whether there is a valid task for a certain id.

There may have been a valid task in the past with the same id but the results may have been deleted from the backend.

回答1:

Celery does not write a state when the task is sent, this is partly an optimization (see http://docs.celeryproject.org/en/latest/userguide/tasks.html#state).

If you really need it, it's simple to add:

from celery import current_app
# `after_task_publish` is available in celery 3.1+
# for older versions use the deprecated `task_sent` signal
from celery.signals import after_task_publish

# when using celery versions older than 4.0, use body instead of headers

@after_task_publish.connect
def update_sent_state(sender=None, headers=None, **kwargs):
    # the task may not exist if sent using `send_task` which
    # sends tasks by name, so fall back to the default result backend
    # if that is the case.
    task = current_app.tasks.get(sender)
    backend = task.backend if task else current_app.backend

    backend.store_result(headers['id'], None, "SENT")

Then you can test for the PENDING state to detect that a task has not (seemingly) been sent:

>>> result.state != "PENDING"

回答2:

AsyncResult.state returns PENDING in case of unknown task ids.

PENDING

Task is waiting for execution or unknown. Any task id that is not known is implied to be in the pending state.

http://docs.celeryproject.org/en/latest/userguide/tasks.html#pending

You can provide custom task ids if you need to distinguish unknown ids from existing ones:

>>> from tasks import add
>>> from celery.utils import uuid
>>> r = add.apply_async(args=[1, 2], task_id="celery-task-id-"+uuid())
>>> id = r.task_id
>>> id
'celery-task-id-b774c3f9-5280-4ebe-a770-14a6977090cd'
>>> if not "blubb".startswith("celery-task-id-"): print "Unknown task id"
... 
Unknown task id
>>> if not id.startswith("celery-task-id-"): print "Unknown task id"
...

回答3:

Right now I'm using following scheme:

Get task id.
Set to memcache key like 'task_%s' % task.id message 'Started'.
Pass task id to client.
Now from client I can monitor task status(set from task messages to memcache).
From task on ready - set to memcache key message 'Ready'.
From client on task ready - start special task that will delete key from memcache and do necessary cleaning actions.

回答4:

You need to call .get() on the AsyncTask object you create to actually fetch the result from the backend.

See the Celery FAQ.

To further clarify on my answer.

Any string is technically a valid ID, there is no way to validate the task ID. The only way to find out if a task exists is to ask the backend if it knows about it and to do that you must use .get().

This introduces the problem that .get() blocks when the backend doesn't have any information about the task ID you supplied, this is by design to allow you to start a task and then wait for its completion.

In the case of the original question I'm going to assume that the OP wants to get the state of a previously completed task. To do that you can pass a very small timeout and catch timeout errors:

from celery.exceptions import TimeoutError
try:
    # fetch the result from the backend
    # your backend must be fast enough to return
    # results within 100ms (0.1 seconds)
    result = AsyncResult('blubb').get(timeout=0.1)
except TimeoutError:
    result = None

if result:
    print "Result exists; state=%s" % (result.state,)
else:
    print "Result does not exist"

It should go without saying that this only work if your backend is storing results, if it's not there's no way to know if a task ID is valid or not because nothing is keeping a record of them.

Even more clarification.

What you want to do cannot be accomplished using the AMQP backend because it does not store results, it forwards them.

My suggestion would be to switch to a database backend so that the results are in a database that you can query outside of the existing celery modules. If no tasks exist in the result database you can assume the ID is invalid.

回答5:

Try

AsyncResult('blubb').state

that may work.

It should return something different.

回答6:

Please correct me if i'm wrong.

if built_in_status_check(task_id) == 'pending'
   if registry_exists(task_id) == true
      print 'Pending'
   else
      print 'Task does not exist'

来源：https://stackoverflow.com/questions/9824172/find-out-whether-celery-task-exists

标签

python

task

status

celery