Clear memory in python loop

问题

How i can clear memory in this python loop ?

import concurrent.futures as futures
with futures.ThreadPoolExecutor(max_workers=100) as executor:
    fs = [executor.submit(get_data, url) for url in link]
    for i, f in enumerate(futures.as_completed(fs)):
        x= (f.result())
        results.append(x)
        del x 
        del f

get_data - simple function wich using requests

回答1:

My solution would be as such:
import concurrent.futures as futures

#split the original grand list into smaller batches  

batchurlList = [grandUrlList[x:x+batchSize] for x in range(0, len(grandUrlList), batchSize)]
for tmpurlList in batchurlList:
    with futures.ThreadPoolExecutor(max_workers=100) as executor:
        myfuture = {executor.submit(myFunction, url): url for url in tmpurlList}
        for future in futures.as_completed(myfuture, timeout=60):
            originalUrl = myfuture[future]
            results.append(future.result())

回答2:

I think I had the same problem recently, The answer is not del but introducing a sleep func... Try;

import time
import concurrent.futures as futures
with futures.ThreadPoolExecutor(max_workers=100) as executor:
    fs = [executor.submit(get_data, url) for url in link]
    for i, f in enumerate(futures.as_completed(fs)):
           x= (f.result())
           results.append(x)
           time.sleep(n_seconds)

Or something like this (I used a while loop over a list of urls)

来源：https://stackoverflow.com/questions/31720674/clear-memory-in-python-loop