Django Celery get task count

问题

I am currently using django with celery and everything works fine.

However I want to be able to give the users an opportunity to cancel a task if the server is overloaded by checking how many tasks are currently scheduled.

How can I achieve this ?

I am using redis as broker.

I just found this : Retrieve list of tasks in a queue in Celery

It is somehow relate to my issue but I don't need to list the tasks , just count them :)

回答1:

If your broker is configured as redis://localhost:6379/1, and your tasks are submitted to the general celery queue, then you can get the length by the following means:

import redis
queue_name = "celery"
client = redis.Redis(host="localhost", port=6379, db=1)
length = client.llen(queue_name)

Or, from a shell script (good for monitors and such):

$ redis-cli -n 1 -h localhost -p 6379 llen celery

回答2:

Here is how you can get the number of messages in a queue using celery that is broker-agnostic.

By using connection_or_acquire, you can minimize the number of open connections to your broker by utilizing celery's internal connection pooling.

celery = Celery(app)

with celery.connection_or_acquire() as conn:
    conn.default_channel.queue_declare(
        queue='my-queue', passive=True).message_count

You can also extend Celery to provide this functionality:

from celery import Celery as _Celery


class Celery(_Celery)

    def get_message_count(self, queue):
        '''
        Raises: amqp.exceptions.NotFound: if queue does not exist
        '''
        with self.connection_or_acquire() as conn:
            return conn.default_channel.queue_declare(
                queue=queue, passive=True).message_count


celery = Celery(app)
num_messages = celery.get_message_count('my-queue')

回答3:

If you have already configured redis in your app, you can try this:

from celery import Celery

QUEUE_NAME = 'celery'

celery = Celery(app)
client = celery.connection().channel().client

length = client.llen(QUEUE_NAME)

回答4:

Get a redis client instance used by Celery, then check the queue length. Don't forget to release the connection every time you use it (use .acquire):

# Get a configured instance of celery:
from project.celery import app as celery_app

def get_celery_queue_len(queue_name):
    with celery_app.pool.acquire(block=True) as conn:
        return conn.default_channel.client.llen(queue_name)

Always acquire a connection from the pool, don't create it manually. Otherwise, your redis server will run out of connection slots and this will kill your other clients.

来源：https://stackoverflow.com/questions/18631669/django-celery-get-task-count

标签

python

django

Redis

celery