celery

I'm fairly new to Celery and I've been attempting setup a simple script to schedule and unschedule tasks. However I feel like I'm running into a weird issue. I have the following setup from celery import Celery app = Celery('celery_test', broker='amqp://', backend='amqp') @app.task def add(x, y): return x + y I start up my celery server just fine and can add tasks. Now when I want to get a list of active tasks things seem to get weird. When I goto use inspect to get a list of scheduled tasks it works exactly once then returns None every time afterwards. >>> i = app.control.inspect() >>> print

The scenario: System running on a server consisting of a Python/Flask web application and background tasks using Celery Both web application and celery workers are run as upstart jobs (Web app behind Nginx) Deployment to production is done with a script that: Stop the upstart jobs Push code to server Run any db migrations Start the upstart jobs How can I enhance the deployment script so it does the following?: Tell the celery worker to stop accepting tasks Wait until any currently running celery tasks are finished Stop the upstart jobs Push code to server Run any db migrations Start the

I've a python celery-redis queue processing uploads and downloads worth gigs and gigs of data at a time. Few of the uploads takes upto few hours. However once such a task finishes, I'm witnessing this bizarre celery behaviour that the celery scheduler is rerunning the just concluded task again by sending it again to the worker (I'm running a single worker) And it just happened 2times on the same task! Can someone help me know why is this happening and how can I prevent it? The tasks are definitely finishing cleanly with no errors reported just that these are extremely long running tasks. I

I have a task in tasks.py like so: @app.task def location(request): .... I am trying to pass the request object directly from a few to task like so: def tag_location(request): tasks.location.delay(request) return JsonResponse({'response': 1}) I am getting an error that it can't be serialized i guess? How do I fix this? trouble is I have file upload objects as well .. its not all simple data types. Because the request object contains references to things which aren't practical to serialize — like uploaded files, or the socket associated with the request — there's no general purpose way to

I have a Django project on an Ubuntu EC2 node, which I have been using to set up an asynchronous using Celery . I am following http://michal.karzynski.pl/blog/2014/05/18/setting-up-an-asynchronous-task-queue-for-django-using-celery-redis/ along with the docs. I've been able to get a basic task working at the command line, using: (env1)ubuntu@ip-172-31-22-65:~/projects/tp$ celery --app=myproject.celery:app worker --loglevel=INFO I just realized, that I have a bunch of tasks in my queue, that had not executed: [2015-03-28 16:49:05,916: WARNING/MainProcess] Restoring 4 unacknowledged message(s).