why my coroutine blocks whole tornado instance?

Question

from tornado import web, gen
import tornado, time

class CoroutineFactorialHandler(web.RequestHandler):
    @web.asynchronous
    @gen.coroutine
    def get(self, n, *args, **kwargs):
        n = int(n)
        def callbacker(iterator, callback):
            try:
                value = next(iterator)
            except StopIteration:
                value = StopIteration
            callback(value)

        def factorial(n):
            x = 1
            for i in range(1, n+1):
                x *= i
                yield

            yield x

        iterator = factorial(n)
        t = time.time()
        self.set_header("Content-Type", "text/plain")
        while True:
            response = yield gen.Task(callbacker, iterator)
            #log.debug("response: %r" %response)
            if response is StopIteration:
                break
            elif response:
                self.write("took : %f sec" %(time.time() - t))
                self.write("\n")
                self.write("f(%d) = %d" %(n, response))

        self.finish()

application = tornado.web.Application([
    (r"^/coroutine/factorial/(?P<n>\d+)", CoroutineFactorialHandler),
    #http://localhost:8888/coroutine/factorial/<int:n>
])

if __name__ == "__main__":
    application.listen(8888)
    ioloop = tornado.ioloop.IOLoop.instance()
    ioloop.start()

21 lines yanked above is the simple factorial calculator. it loops N times, in generator fashion.

the problem is, when this code is executing it blocks the whole tornado.

what I want to achieve is writing some helper for tornado that treats generators as coroutine, and therefore can serve requests in asynchronous manner. (I have read Using a simple python generator as a co-routine in a Tornado async handler?)

why does the simple increase-and-multiply-by-n loop block the whole tornado?

edit : I edited the code to include the whole application, that you can run and test it. I'm running tornado 3.1.1 on python 2.7

Answer 1

You have to remember that Tornado runs in one thread. The code is split into task that are called sequentially in main loop. If one of these task takes long to finish (because of blocking functions like time.sleep() or some heavy computation like factorial) it will block entire loop as a result.

So what you can do...? One solution is to create loop using IOLoop.add_callback():

from tornado import web, gen
import tornado, time

class CoroutineFactorialHandler(web.RequestHandler):
    def factorial(self, limit=1):
        count = 1
        fact = 1
        while count <= limit:
            yield fact
            count = count + 1
            fact = fact * count 

    def loop(self):
        try:
            self.fact = self.generator.next()
            tornado.ioloop.IOLoop.instance().add_callback(self.loop)
        except StopIteration:
            self.write("took : %f sec" %(time.time() - self.t))
            self.write("\n")
            self.write("f(%d) = %d" % (self.n, self.fact))
            self.finish()

    @web.asynchronous
    def get(self, n, *args, **kwargs):
        self.n = int(n)
        self.generator = self.factorial(self.n)
        self.t = time.time()
        self.set_header("Content-Type", "text/plain")
        tornado.ioloop.IOLoop.instance().add_callback(self.loop)

application = tornado.web.Application([
    (r"^/coroutine/factorial/(?P<n>\d+)", CoroutineFactorialHandler),
    #http://localhost:8888/coroutine/factorial/<int:n>
])

if __name__ == "__main__":
    application.listen(8888)
    ioloop = tornado.ioloop.IOLoop.instance()
    ioloop.start()

Every multiplication is a separate task here, which allows mixing factorial generator calls from different requests. This is a good approach if every call to generator took same amount of time. However if you will be computing 100000! then at some point in time tasks in sequence will be looking like 90000!*90001, 90001!*90002 and so on. It takes some time to have this computed even if its only one multiplication instead of whole loop so the other request will be delayed. For such big input integer you have to make computations in another thread to have fair share of processor time for a request. Here is example how to do this: http://lbolla.info/blog/2013/01/22/blocking-tornado

As a side note, in factorial you have a lot of redundancy so you should keep list of solutions for some n at memory to turn them back instantly without wasting processor time for same computation over and over again.