Increasing throughput in a python script

Question

I\'m processing a list of thousands of domain names from a DNSBL through dig, creating a CSV of URLs and IPs. This is a very time-consuming process that can take several hours.

Answer 1

In order to keep pace with the server updates, one must take less than 15 minutes to execute. Does your script take 15 minutes to run? If it doesn't take 15 minutes, you're done!

I would investigate caching and diffs from previous runs in order to increase performance.