django-celery

When I run syncdb, I notice a lot of tables created like: djcelery_crontabschedule ... djcelery_taskstate django-kombu is providing the transport, so it can't be related to the actual queue. Even when I run tasks, I still see nothing populated in these tables. What are these tables used for? Monitoring purposes only -- if I enable it? If so, is it also true that if I do a lookup of AsyncResult(), I'm guessing that is actually looking up the task result via the django-kombu tables instead of djcelery? Thanks. Mauro Rocco The celery task_state table, populated by the daemon celerycam, is just

I keep receiving this error... [2012-06-14 11:54:50,072: ERROR/MainProcess] Hard time limit (300s) exceeded for movies.tasks.encode_media[14cad954-26e2-4511-94ec-b17b9a4149bb] [2012-06-14 11:54:50,111: ERROR/MainProcess] Task movies.tasks.encode_media[bc173429-77ae-4c96-b987-75337f915ec5] raised exception: TimeLimitExceeded(300,) Traceback (most recent call last): File "/srv/virtualenvs/filmlib/local/lib/python2.7/site-packages/celery/concurrency/processes/pool.py", line 370, in _on_hard_timeout raise TimeLimitExceeded(hard_timeout) TimeLimitExceeded: 300 even though I have CELERYD_TASK_TIME

I am using Celery and MongoEngine as part of my Django App with. I am getting this warning, when a celery @shared_task accesses the mongodb database via mongoengine model classes: UserWarning: MongoClient opened before fork. Create MongoClient with connect=False,or create client after forking. See PyMongo's documentation for details: http://api.mongodb.org/python/current/faq.html#using-pymongo-with-multiprocessing It clearly has something to do with multiprocessing and pyMongo that is that mongoengine is based on. My question is: What is the best strategy to avoid this issue with mongoengine?

I'm trying to apply a decorator after the celery @task decorator, something like. @send_email @task def any_function(): print "inside the function" I can get it to work in the way it is recommended in the docs, i.e. to put the decorator before the task decorator, but in this case I would like to access the task instance in my decorator. The @send_email would have to be a class decorator, this is what I tried without success: class send_email(object): ''' wraps a Task celery class ''' def __init__(self, obj): self.wrapped_obj = obj functools.update_wrapper(self, obj) def __call__(self, *args, *

I'm trying to understand how and when tasks are cleaned up in celery. From looking at the task docs I see that: Old results will be cleaned automatically, based on the CELERY_TASK_RESULT_EXPIRES setting. By default this is set to expire after 1 day: if you have a very busy cluster you should lower this value. But this quote is from the RabbitMQ Result Backend section and I do not see any similar text in the Database Backend section. So my question is: is there a backend agnostic approach I can take for old task clean-up with celery and if not is there a DB Backend specific approach I should

I've been using flower locally and it seems easy enough to setup and run, but I can't see how I would set it up in a production environment. In particular, how can I add authentication and how would I define a url to access it? For custom address, use the --address flag. For auth, use the --basic_auth flag. See below: # celery flower --help Usage: /usr/local/bin/celery [OPTIONS] Options: --address run on the given address --auth regexp of emails to grant access --basic_auth colon separated user-password to enable basic auth --broker_api inspect broker e.g. http://guest:guest@localhost:15672