Send Simultaneous Requests python (all at once)

Question

I\'m trying to create a script that send\'s over 1000 requests to one page at the same time. But requests library with threading (1000) threads. Seems to be doing to first 5

Answer 1

I have generally found that the best solution is to use an asynchronous library like tornado. The easiest solution that I found however is to use ThreadPoolExecutor.

---

import requests
from concurrent.futures import ThreadPoolExecutor

def get_url(url):
    return requests.get(url)
with ThreadPoolExecutor(max_workers=50) as pool:
    print(list(pool.map(get_url,list_of_urls)))