I have a Python program that uses the \"threading\" module. Once every second, my program starts a new thread that fetches some data from the web, and stores this data to my
Scrapy seems like a potential answer to my question. Its home page describes my exact task. (Though I'm not sure how stable the code is yet.)