Python asyncio.semaphore in async-await function

Question

I am trying to teach myself Python\'s async functionality. To do so I have built an async web scraper. I would like to limit the total number of connections I have open at o

Answer 1

OK, so this is really silly but I just replaces yield from with await in the semaphore context manager and it is working perfectly.

sema = asyncio.BoundedSemaphore(5)

async def get_page_text(url):
    with (await sema):
        try:
            resp = await aiohttp.request('GET', url)
            if resp.status == 200:
                ret_val = await resp.text()
        except:
            raise ValueError
        finally:
            await resp.release()
    return ret_val

Answer 2

You can use the async with statement to get an asynchronous context manager:

#!/usr/local/bin/python3.5
import asyncio
from aiohttp import ClientSession


sema = asyncio.BoundedSemaphore(5)

async def hello(url):
    async with ClientSession() as session:
        async with sema, session.get(url) as response:
            response = await response.read()
            print(response)

loop = asyncio.get_event_loop()
loop.run_until_complete(hello("http://httpbin.org/headers"))

Example taken from here. The page is also a good primer for asyncio and aiohttp in general.