How to limit concurrency with Python asyncio?

后端 未结 5 1237
我在风中等你
我在风中等你 2020-11-27 02:35

Let\'s assume we have a bunch of links to download and each of the link may take a different amount of time to download. And I\'m allowed to download using utmost 3 connecti

5条回答
  •  萌比男神i
    2020-11-27 03:10

    I used Mikhails answer and ended up with this little gem

    async def gather_with_concurrency(n, *tasks):
        semaphore = asyncio.Semaphore(n)
    
        async def sem_task(task):
            async with semaphore:
                return await task
        return await asyncio.gather(*(sem_task(task) for task in tasks))
    

    Which you would run instead of normal gather

    await gather_with_concurrency(100, *my_coroutines)
    

提交回复
热议问题