python asyncio - how to wait for a cancelled shielded task?

Question

If I have a coroutine which runs a task which should not be cancelled, I will wrap that task in asyncio.shield().

It seems the behavior of cancel

Answer 1

It seems the behavior of cancel and shield is not what I would expect. If I have a task wrapped in shield and I cancel it, the await-ing coroutine returns from that await statement immediately rather than awaiting for the task to finish as shield would suggest. Additionally, the task that was run with shield continues to run but its future is now cancelled an not await-able.

Conceptually shield is like a bullet-proof vest that absorbs the bullet and protects the wearer, but is itself destroyed by the impact. shield absorbs the cancellation, and reports itself as canceled, raising a CancelledError when asked for result, but allows the protected task to continue running. (Artemiy's answer explains the implementation.)

Cancellation of the future returned by shield could have been implemented differently, e.g. by completely ignoring the cancel request. The current approach ensures that the cancellation "succeeds", i.e. that the canceller can't tell that the cancellation was in fact circumvented. This is by design, and it makes the cancellation mechanism more consistent on the whole.

What is the proper method to shield a task from cancellation and then wait for it to complete before returning

By keeping two objects: the original task, and the shielded task. You pass the shielded task to whatever function it is that might end up canceling it, and you await the original one. For example:

async def coro():
    print('starting')
    await asyncio.sleep(2)
    print('done sleep')

async def cancel_it(some_task):
    await asyncio.sleep(0.5)
    some_task.cancel()
    print('cancellation effected')

async def main():
    loop = asyncio.get_event_loop()
    real_task = loop.create_task(coro())
    shield = asyncio.shield(real_task)
    # cancel the shield in the background while we're waiting
    loop.create_task(cancel_it(shield))
    await real_task

    assert not real_task.cancelled()
    assert shield.cancelled()

asyncio.get_event_loop().run_until_complete(main())

The code waits for the task to fully complete, despite its shield getting cancelled.