celery-task

I have setup celery + rabbitmq for on a 3 cluster machine. I have also created a task which generates a regular expression based on data from the file and uses the information to parse text. from celery import Celery celery = Celery('tasks', broker='amqp://localhost//') import re @celery.task def add(x, y): return x + y def get_regular_expression(): with open("text") as fp: data = fp.readlines() str_re = "|".join([x.split()[2] for x in data ]) return str_re @celery.task def analyse_json(tw): str_re = get_regular_expression() re.match(str_re,tw.text) I can make the call to this task very easily

this is the code which i am running: from __future__ import absolute_import from celery import Celery celery1 = Celery('celery',broker='amqp://',backend='amqp://',include=['tasks']) celery1.conf.update( CELERY_TASK_RESULT_EXPIRES=3600, ) if __name__ == '__main__': celery1.start() when i execute the above code it gives me the following error: ImportError: cannot import name Celery Jaredp37 I ran into this same error as well and renaming the file fixed it. For anyone else encountering this, the reason WHY you get this issue, your local celery.py is getting imported instead of the actual celery

Under some conditions, I want to make a celery task fail from within that task. I tried the following: from celery.task import task from celery import states @task() def run_simulation(): if some_condition: run_simulation.update_state(state=states.FAILURE) return False However, the task still reports to have succeeded: Task sim.tasks.run_simulation[9235e3a7-c6d2-4219-bbc7-acf65c816e65] succeeded in 1.17847704887s: False It seems that the state can only be modified while the task is running and once it is completed - celery changes the state to whatever it deems is the outcome (refer to this

Update for the bounty I'd like a solution that does not involve a monitoring thread, if possible. I know I can view scheduled and active tasks using the Inspect class of my apps Control . i = myapp.control.inspect() currently_running = i.active() scheduled = i.scheduled() But I could not find any function to show already finished tasks. I know that this information mus be at least temporarily accessible, because I can look up a finished task by its task_id : >>> r = my task.AsyncResult(task_id=' ... ') >>> r.state u'SUCCESS' How can I get a complete list of scheduled, active and finished tasks

I'm using celery and django-celery. I have defined a periodic task that I'd like to test. Is it possible to run the periodic task from the shell manually so that I view the console output? Have you tried just running the task from the Django shell? You can use the .apply method of a task to ensure that it is run eagerly and locally. Assuming the task is called my_task in Django app myapp in a tasks submodule: $ python manage.py shell >>> from myapp.tasks import my_task >>> eager_result = my_task.apply() The result instance has the same API as the usual AsyncResult type, except that the result

In Celery, I want to start a task only when all the other tasks have completed. I found some resources like this one : Celery Starting a Task when Other Tasks have Completed and Running a task after all tasks have been completed But I am quite new to celery and could not really understand the above (or many other resources for that matter). So I have defined a task as so in a tasks.py : @celapp.task() def sampleFun(arg1, arg2, arg3): # do something here and I call it as this : for x in xrange(4): tasks.sampleFun.delay(val1, val2, val3) And I assume that there would be 4 different tasks created

While I tried to start the worker I got a issue: ImportError: No module named 'project' Traceback (most recent call last): File "/usr/local/bin/celery", line 11, in <module> sys.exit(main()) File "/usr/local/lib/python3.5/dist-packages/celery/__main__.py", line 16, in main _main() File "/usr/local/lib/python3.5/dist-packages/celery/bin/celery.py", line 322, in main cmd.execute_from_commandline(argv) File "/usr/local/lib/python3.5/dist-packages/celery/bin/celery.py", line 496, in execute_from_commandline super(CeleryCommand, self).execute_from_commandline(argv))) File "/usr/local/lib/python3.5