RabbitMQ on EC2 Consuming Tons of CPU

Question

I am trying to get RabbitMQ with Celery and Django going on an EC2 instance to do some pretty basic background processing. I\'m running rabbitmq-server 2.5.0 on a large EC2

Answer 1

To add to Eric Conner's solution to his own problem, http://docs.celeryproject.org/en/latest/userguide/tasks.html#tips-and-best-practices states:

Ignore results you don’t want

If you don’t care about the results of a task, be sure to set the ignore_result option, as storing results wastes time and resources.

@app.task(ignore_result=True)
def mytask(…):
    something()

Results can even be disabled globally using the CELERY_IGNORE_RESULT setting.

That along with Eric's answer is probably a bare minimum best practices for managing your results backend.

If you don't need a results backend, set CELERY_IGNORE_RESULT or don't set a results backend at all. If you do need a results backend, set CELERY_AMQP_TASK_RESULT_EXPIRES to be safeguarded against unused results building up. If you don't need it for a specific app, set the local ignore as above.