How do I monitor how busy a Python event loop is?

Question

I have an asynchronous application that is serving requests via aiohttp and doing other async tasks (interacting with a database, handling messages, making requests itself a

Answer 1

One way of measuring how much time the CPU was busy (applicable for all sorts of IO-bound situations, not just async) is to periodically check time.process_time().

For example, here's how I would implement something which occasionally prints the percentage of time that the loop was "busy" because the process was running code on the CPU:

import asyncio
import time

async def monitor():
    while True:
        before = time.process_time()
        await asyncio.sleep(10)
        after = time.process_time()
        print('{:.2%}'.format((after - before) / 10))


async def busy_wait():
    while True:
        await asyncio.sleep(.001)

loop = asyncio.get_event_loop()
loop.create_task(monitor())
loop.create_task(busy_wait())
loop.run_forever()

The monitor coroutine measures the amount of process time elapsed every 10 seconds, and prints it as a percentage of the full 10 seconds.

The busy_wait coroutine generates artificial load on the CPU by repeatedly sleeping for short periods. By increasing the length of the sleep you can make the process time ratio decrease arbitrarily, and by decreasing the length of the sleep to 0 you can make the process time approach 100% of the wall-clock time.

One caveat: this will only tell you how busy the python event loop is in terms of time spent "running python code" (in the CPU-bound sense). If someone is blocking the event loop with calls to time.sleep() (rather than asyncio.sleep()), my method will show the loop as being free but it is really "busy" in the sense that it is blocked by a system-level sleep. Most correctly written async code shouldn't call time.sleep() or do blocking IO, but if it does, it will not be reflected correctly by this type of monitoring.