django-celery

How to use Django with AWS Elastic Beanstalk that would also run tasks by celery on main node only? This is how I set up celery with django on elastic beanstalk with scalability working fine. Please keep in mind that 'leader_only' option for container_commands works only on environment rebuild or deployment of the App. If service works long enough, leader node may be removed by Elastic Beanstalk. To deal with that, you may have to apply instance protection for your leader node. Check: http://docs.aws.amazon.com/autoscaling/latest/userguide/as-instance-termination.html#instance-protection

Is there a way to determine if any task is lost and retry it? I think that the reason for lost can be dispatcher bug or worker thread crash. I was planning to retry them but I'm not sure how to determine which tasks need to be retired? And how to make this process automatically? Can I use my own custom scheduler which will create new tasks? Edit: I found from the documentation that RabbitMQ never loose tasks, but what happens when worker thread crash in the middle of task execution? What you need is to set CELERY_ACKS_LATE = True Late ack means that the task messages will be acknowledged after

For a task like this: from celery.decorators import task @task() def add(x, y): if not x or not y: raise Exception("test error") return self.wait_until_server_responds( if it throws an exception and I want to retry it from the daemon side, how can apply an exponential back off algorithm, i.e. after 2^2, 2^3,2^4 etc seconds? Also is the retry maintained from the server side, such that if the worker happens to get killed then next worker that spawns will take the retry task? asksol The task.request.retries attribute contains the number of tries so far, so you can use this to implement

I'm working on a project using django and celery(django-celery). Our team decided to wrap all data access code within (app-name)/manager.py (NOT wrap into Managers like the django way), and let code in (app-name)/task.py only dealing with assemble and perform tasks with celery(so we don't have django ORM dependency in this layer). In my manager.py , I have something like this: def get_tag(tag_name): ctype = ContentType.objects.get_for_model(Photo) try: tag = Tag.objects.get(name=tag_name) except ObjectDoesNotExist: return Tag.objects.none() return tag def get_tagged_photos(tag): ctype =

I'm trying to use the methods of class as the django-celery tasks, marking it up using @task decorator. The same situation is discribed here , asked by Anand Jeyahar. It's something like this class A: @task def foo(self, bar): ... def main(): a = A() ... # what i need a.foo.delay(bar) # executes as celery task a.foo(bar) # executes locally The problem is even if i use class instance like this a.foo.delay(bar) it says, that foo needs at least two arguments, which meens that self pointer misses. More information: I can't convert class to module because of inheritance Methods are strongly

I'm trying to get Celery logging working with Django . I have logging set-up in settings.py to go to console (that works fine as I'm hosting on Heroku ). At the top of each module, I have: import logging logger = logging.getLogger(__name__) And in my tasks.py, I have: from celery.utils.log import get_task_logger logger = get_task_logger(__name__) That works fine for logging calls from a task and I get output like this: 2012-11-13T18:05:38+00:00 app[worker.1]: [2012-11-13 18:05:38,527: INFO/PoolWorker-2] Syc feed is starting But if that task then calls a method in another module, e.g. a