Best python multithreading solution for processing millions of files independently?

Question

I will have potentially millions of files sitting on variolous places. These files comes from different data vendor and contain stock trading information. That being said, i