invoke celery task from tornado [duplicate]

问题:

This question already has an answer here:

• Tornado celery integration hacks 4 answers

How can someone invoke a celery task from tornado, and get the result via a callback?

This post claims that someone must simply put a message via RabbitMQ and then the task shall be executed. This makes sense, but can someone give an example in python (even better in tornado, with a callback)? Personally, I use mongodb as my message broker, but I can switch to Redis or RabbitMQ as well..

EDIT: To clarify things, I want an example with a callback. For example, this tornado code

TestTask.delay(callback = self._on_celery_response)  ... def _on_celery_response(self, result):     print "hello from _on_celery_repsonse" , result 

does not work. My TestTask is:

class TestTask(Task):     name = "tornadoServer.Test"     def run(self, callback=None,  **kwargs):         result = {'result': "hello from celery task invoked by tornado"}         if callback is not None:             subtask(callback).delay(result)         return result 

and the traceback:

    File "/home/hymloth/Desktop/DJANGO/NOO1/tornadoServer/tornado/stack_context.py", line 183, in wrapped     callback(*args, **kwargs)   File "/home/hymloth/Desktop/DJANGO/NOO1/tornadoServer/asyncmongo/connection.py", line 183, in _parse_response     callback(response)   File "/home/hymloth/Desktop/DJANGO/NOO1/tornadoServer/asyncmongo/cursor.py", line 399, in _handle_response     orig_callback(result['data'], error=None)   File "/home/hymloth/Desktop/DJANGO/NOO1/tornadoServer/basic_auth_handlers.py", line 66, in _on_response     celery_tasks.TestTask.delay(self._on_celery_response)   File "/usr/local/lib/python2.6/dist-packages/celery-2.2.7-py2.6.egg/celery/task/base.py", line 338, in delay     return self.apply_async(args, kwargs)   File "/usr/local/lib/python2.6/dist-packages/celery-2.2.7-py2.6.egg/celery/task/base.py", line 460, in apply_async     **options)   File "/usr/local/lib/python2.6/dist-packages/celery-2.2.7-py2.6.egg/celery/app/amqp.py", line 230, in delay_task     send(body, exchange=exchange, **extract_msg_options(kwargs))   File "/usr/local/lib/python2.6/dist-packages/kombu-1.1.6-py2.6.egg/kombu/compat.py", line 101, in send     return self.publish(*args, **kwargs)   File "/usr/local/lib/python2.6/dist-packages/kombu-1.1.6-py2.6.egg/kombu/messaging.py", line 124, in publish     compression, headers)   File "/usr/local/lib/python2.6/dist-packages/kombu-1.1.6-py2.6.egg/kombu/messaging.py", line 147, in _prepare     body) = encode(body, serializer=serializer)   File "/usr/local/lib/python2.6/dist-packages/kombu-1.1.6-py2.6.egg/kombu/serialization.py", line 119, in encode     payload = encoder(data)   File "/usr/lib/python2.6/copy_reg.py", line 70, in _reduce_ex     raise TypeError, "can't pickle %s objects" % base.__name__ TypeError: can't pickle instancemethod objects 

The Task works ok without the callback.. Any suggestions?

回答1:

The callback object should be a celery Task too or your code doesn't work.

You can use signals inside the task body if your callback function don't have to be a celery task.

http://docs.python.org/library/signal.html

回答2:

If you have the module with your tasks in the pythonpath of your tornado process a simple:

my_task.delay() 

should work as long as celery is configured properly in your tornado instance. To do that when starting your server pass:

CELERY_CONFIG_MODULE=app.foo.celeryconfig 

in the environment so when you do import celery, celery set up would know where your settings for this instance are.

回答3:

celery has callback function, you could go to http://ask.github.com/celery/userguide/tasksets.html

from celery.task import task from celery.task.sets import subtask  @task def add(x, y, callback=None):     result = x + y      if callback is not None:         subtask(callback).delay(result)      return result 

文章来源: invoke celery task from tornado [duplicate]