Python 3 asyncio - yield from vs asyncio.async stack usage

Question

I'm evaluating different patterns for periodic execution (actual sleep/delays ommited for brevity) using the Python 3 asyncio framework, and I have two pieces of code that behave diffrently and I can't explain why. The first version, which uses yield from to call itself recursively exhausts the stack in about 1000 iterations, as I expected. The second version calls the coroutine recursively, but delegates actual event loop execution to asyncio.async and doesn't exhaust the stack. Can you explain in detail why the stack isn't being used by the second version? What are the differences between the two ways of executing this coroutine?

First version (yield from):

@asyncio.coroutine
def call_self(self, i):
    print('calling self', i)
    yield from self.call_self(i + 1)

Second version (asyncio.async):

@asyncio.coroutine
def call_self(self, i):
    print('calling self', i)
    asyncio.async(self.call_self(i + 1))

Answer 1

The first example, using yield from, actually blocks each instance of call_self until its recursive call to call_self returns. This means the call stack keeps growing until you run out of stack space. As you mentioned, this is the obvious behavior.

The second example, using asyncio.async, doesn't block anywhere. So, each instance of call_self immediately exits after running asyncio.async(...), which means the stack doesn't grow infinitely, which means you don't exhaust the stack. Instead, asyncio.async schedules call_self gets to be executed on the iteration of the event loop, by wrapping it in a asyncio.Task.

Here's the __init__ for Task:

def __init__(self, coro, *, loop=None):
    assert iscoroutine(coro), repr(coro)  # Not a coroutine function!
    super().__init__(loop=loop)
    self._coro = iter(coro)  # Use the iterator just in case.
    self._fut_waiter = None
    self._must_cancel = False
    self._loop.call_soon(self._step)  # This schedules the coroutine to be run
    self.__class__._all_tasks.add(self)

The call to self._loop.call_soon(self._step) is what actually makes the coroutine execute. Because it's happening in a non-blocking way, the call stack from call_self never grows beyond the call to the Task constructor. Then the next instance of call_self gets kicked off by the event loop on its next iteration (which starts as soon as the previous call_self returns, assuming nothing else is running in the event loop), completely outside of the context of the previous call_self instance.