How to wrap asynchronous and gen functions together in Tornado?

Question

How to wrap asynchronous and gen functions together in Tornado? My code looks like below, the error is 'Future' object has no attribute 'body'.

Did I place the decorators in a wrong way?

import tornado.httpclient
import tornado.web
import tornado.gen
import tornado.httpserver
import tornado.ioloop

class Class1(tornado.web.RequestHandler):

    @tornado.web.asynchronous
    def post(self, *args, **kwargs):
        url = self.get_argument('url', None)
        response = self.json_fetch('POST', url, self.request.body)
        self.write(response.body)
        self.finish()

    @tornado.gen.engine
    def json_fetch(self, method, url, body=None, *args, **kwargs):
        client = tornado.httpclient.AsyncHTTPClient()
        headers = tornado.httputil.HTTPHeaders({"content-type": "application/json charset=utf-8"})
        request = tornado.httpclient.HTTPRequest(url, method, headers, body)
        yield tornado.gen.Task(client.fetch, request)

Answer 1

You don't need "asynchronous" in this code example. "gen.engine" is obsolete, use "coroutine" instead. You don't generally need to use "gen.Task" much these days, either. Make four changes to your code:

1. Wrap "post" in "coroutine"
2. "yield" the result of self.json_fetch instead of using the result directly.
3. No need to call "finish" in a coroutine, Tornado finishes the response when a coroutine completes.
4. Wrap json_fetch in "coroutine", too.

The result:

class ClubCreateActivity(tornado.web.RequestHandler):

    @tornado.gen.coroutine
    def post(self, *args, **kwargs):
        url = self.get_argument('url', None)
        response = yield self.json_fetch('POST', url, self.request.body)
        self.write(response.body)

    @tornado.gen.coroutine
    def json_fetch(self, method, url, body=None, *args, **kwargs):
        client = tornado.httpclient.AsyncHTTPClient()
        headers = tornado.httputil.HTTPHeaders({"content-type": "application/json charset=utf-8"})
        request = tornado.httpclient.HTTPRequest(url, method, headers, body)
        response = yield client.fetch(request)
        raise gen.Return(response)

Further reading:

• The section on coroutines in the Tornado User's Guide
• Tornado async request handlers
• My article on refactoring coroutines

Answer 2

The recommended method in tornado official documentation is using @tornado.gen.coroutine and yield together.

If you want to use both asynchronous and advantage of yield, you should nesting @tornado.web.asynchronous decorator followed by @tornado.gen.engine

Answer 3

Documentation about "asynchronous call own function" but without additional external callback-function — Asynchronous and non-Blocking I/O

You can make your json_fetch like this:

from tornado.concurrent import Future

def json_fetch(self, method, url, body=None, *args, **kwargs):
    http_client = tornado.httpclient.AsyncHTTPClient()
    my_future = Future()
    fetch_future = http_client.fetch(url)
    fetch_future.add_done_callback(
        lambda f: my_future.set_result(f.result()))
    return my_future

Or like this (from A. Jesse Jiryu Davis's answer):

from tornado import gen

@gen.coroutine
def json_fetch(self, method, url, body=None, *args, **kwargs):
    http_client = tornado.httpclient.AsyncHTTPClient()
    headers = tornado.httputil.HTTPHeaders({"content-type": "application/json charset=utf-8"})
    request = tornado.httpclient.HTTPRequest(url, method, headers, body)
    response = yield http_client.fetch(request)
    raise gen.Return(response)

* wrap "post" in "gen.coroutine" and "yield" call of json_fetch.

** "raise gen.Return(response)" for Python2 only, in Python3.3 and later you should write "return response".

Thanks to A. Jesse Jiryu Davis for link "Tornado async request handlers", "Asynchronous and non-Blocking I/O" was found there.