I\'m trying to make a bunch of requests (~1000) using Asyncio and the aiohttp library, but I am running into a problem that I can\'t find much info on.
When I run th
The bug is filed as https://github.com/python/asyncio/issues/258 Stay tuned.
As quick workaround I suggest using custom executor, e.g.
loop = asyncio.get_event_loop()
executor = concurrent.futures.ThreadPoolExecutor(5)
loop.set_default_executor(executor)
Before finishing your program please do
executor.shutdown(wait=True)
loop.close()
You're right, loop.getaddrinfo uses a ThreadPoolExecutor
to run socket.getaddrinfo
in a thread.
You're using asyncio.wait_for with a timeout, which means res = yield from asyncio.wait_for...
will raise a asyncio.TimeoutError
after 4 seconds. Then the get_status
coroutines return None
and the loop stops. If a job finishes after that, it will try to schedule a callback in the event loop and raises an exception since it is already closed.