python asyncio add_done_callback with async def

Question

I have 2 functions: The first one, def_a, is an asynchronous function and the second one is def_b which is a regular function and called with the result of def_a as a callback with the add_done_callback function.

My code looks like this:

import asyncio

def def_b(result):
    next_number = result.result()
    # some work on the next_number
    print(next_number + 1)

async def def_a(number):
    await some_async_work(number)
    return number + 1

loop = asyncio.get_event_loop()
task = asyncio.ensure_future(def_a(1))
task.add_done_callback(def_b)
response = loop.run_until_complete(task)
loop.close()

And it's work perfectly.

The problem began when also the second function, def_b, became asynchronous. Now it looks like this:

async def def_b(result):
    next_number = result.result()
    # some asynchronous work on the next_number
    print(next_number + 1)

But now I can not provide it to the add_done_callback function, because it's not a regular function.

My question is- Is it possible and how can I provide def_b to the add_done_callback function if def_b is asynchronous?

Answer 1

add_done_callback is considered a "low level" interface. When working with coroutines, you can chain them in many ways, for example:

import asyncio


async def my_callback(result):
    print("my_callback got:", result)
    return "My return value is ignored"


async def coro(number):
    await asyncio.sleep(number)
    return number + 1


async def add_success_callback(fut, callback):
    result = await fut
    await callback(result)
    return result


loop = asyncio.get_event_loop()
task = asyncio.ensure_future(coro(1))
task = add_success_callback(task, my_callback)
response = loop.run_until_complete(task)
print("response:", response)
loop.close()

Keep in mind add_done_callback will still call the callback if your future raises an exception (but calling result.result() will raise it).

Answer 2

This only works for one future job, if you have multiple async jobs, they will blocks each other, a better way is using asyncio.as_comleted() to iterate future list:

import asyncio

async def __after_done_callback(future_result):
    # await for something...
    pass

async def __future_job(number):
    await some_async_work(number)
    return number + 1

loop = asyncio.get_event_loop()
tasks = [asyncio.ensure_future(__future_job(x)) for x in range(100)]  # create 100 future jobs

for f in asyncio.as_completed(tasks):
    result = await f
    await __after_done_callback(result)

loop.close()