Send Simultaneous Requests python (all at once)

Question

I\'m trying to create a script that send\'s over 1000 requests to one page at the same time. But requests library with threading (1000) threads. Seems to be doing to first 5

Answer 1

I know this is an old question, but you can now do this using asyncio and aiohttp.

import asyncio
import aiohttp
from aiohttp import ClientSession

async def fetch_html(url: str, session: ClientSession, **kwargs) -> str:
    resp = await session.request(method="GET", url=url, **kwargs)
    resp.raise_for_status()
    return await resp.text()

async def make_requests(url: str, **kwargs) -> None:
    async with ClientSession() as session:
        tasks = []
        for i in range(1,1000):
            tasks.append(
                fetch_html(url=url, session=session, **kwargs)
            )
        results = await asyncio.gather(*tasks)
        # do something with results

if __name__ == "__main__":
    asyncio.run(make_requests(url='http://test.net/'))

You can read more about it and see an example here.