Using Tornado and Twisted at the same time

Question

I am in a weird situation where I have to use Twisted in a system built completely out of Tornado. They can share the same IOLoop so I know they can work together. My question is can I safely use their co-routine decorators in the same function? For example:

import tornado.platform.twisted
tornado.platform.twisted.install()

...

@gen.engine
@defer.inlineCallbacks
def get(self):

    ...

    a = yield gen.Task(getA) # tornado
    b = yield proxy.callRemote(getB) # twisted

    ...

    defer.returnValue(a + b) # twisted

They do work on the same IOLoop so I am thinking this should be fine. Would there be any unforeseen consequences? Thanks in advance.

Answer 1

No, this wouldn't work. In your case inlineCallbacks is wrapped directly around your generator and gen.engine is wrapped outside. The problem is that inlineCallbacks does not know anything about gen.Task and it will yield it immediately (it has no way of passing it along to gen.engine).

To elaborate: if you yield obj inside an inlineCallbacks-wrapped generator, two things can happen:

1. obj is a Deferred in which case control is returned to the reactor until that Deferred fires.
2. obj is something else, in which case it is immediately sent back into your generator.

In your case, the result would be:

a = yield gen.Task(getA) # continues right through
# a is of type gen.Task here
b = yield proxy.callRemote(getB) # waits for result of proxy.callRemote

See here for how inlineCallbacks is implemented.

What is the right way to do this? Try to use either inlineCallbacks or gen.engine (but not both). Wrap the alien gen.Task (or Deferred) into the "native" form. I am not familiar with Tornado but maybe this question helps.

Alternatively, write your own decorator like inlineCallbacks that handles gen.Task as well.

Answer 2

Looks like what you want is Cyclone, a web server framework for Python that implements the Tornado API as a Twisted protocol.