Terminate multiple threads when any thread completes a task

Question

I am new to both python, and to threads. I have written python code which acts as a web crawler and searches sites for a specific keyword. My question is, how can I use thre

Answer 1

First of all, threading is not a solution in Python. Due to GIL, Threads does not work in parallel. So you can handle this with multiprocessing and you'll be limited with the number of processor cores.

What's the goal of your work ? You want to have a crawler ? Or you have some academic goals (learning about threading and Python, etc.) ?

Another point, Crawl waste more resources than other programs, so what is the sale your crawl ?