AsyncResult(task_id) returns “PENDING” state even after the task started

问题

In the project, I try to poll task.state of a long running task and update its running status. It worked in the development, but it won't work when I move the project on production server. I kept getting 'PENDING' even I can see the task started on flower. However, I can still get the results updated when the task finished, which when task.state == 'SUCCESS'. I use python 2.6, Django 1.6 and Celery 3.1 in the production, result backend AMQP.

@csrf_exempt
def poll_state(request):
    data = 'Fail'

    if request.is_ajax():
            if 'task_id' in request.POST.keys() and request.POST['task_id']:
                    task_id = request.POST['task_id']
                    email = request.POST['email']
                    task = AsyncResult(task_id)
                    print "task.state=", task.state
                    if task.state == 'STARTED':
                            task_state = 'Running'
                            data = 'Running'
                            #data = 'Running'
                    elif task.state == 'PENDING' or task.state == 'RETRY':
                            task_state = 'Waiting'
                            data = 'Pending'
                    elif task.state == 'SUCCESS':
                            task_state = 'Finished'
                            if task.result:
                                    data = task.result
                            else:
                                    data = 'None'

                    else:
                            task_state = task.state
                            data = 'Error'
                            print 'data status =', task_state
            else:
                    task_state = task.state
                    data = 'Error'
    else:
            task_state = task.state
            data = "Error"

    json_data = json.dumps({'task_state':task_state, 'task_data':data})

 return HttpResponse(json_data, mimetype='application/json')

on another note, flower always show the workers' status offline, but tasks status were correct. When using celery events 3.1.12 (Cipater), it shows correct worker status.

回答1:

It's probably related to CELERY_TRACK_STARTED setting. Quoting the docs:

CELERY_TRACK_STARTED

If True the task will report its status as “started” when the task is executed by a worker. The default value is False as the normal behaviour is to not report that level of granularity. Tasks are either pending, finished, or waiting to be retried. Having a “started” state can be useful for when there are long running tasks and there is a need to report which task is currently running.

Maybe you have CELERY_TRACK_STARTED = True in your development settings, but not in production ?