concurrent.futures

My setup is python tornado server, which asynchronously processes tasks with a ThreadPoolExecutor . In some conditions, the task might turn into infinite loop. With the with_timeout decorator, I have managed to catch the timeout exception and return an error result to the client. The problem is that the task is still running in the background. How it is possible to stop the task from running in the ThreadPoolExecutor ? Or is it possible to cancel the Future ? Here is the code that reproduces the problem. Run the code with tornado 4 and concurrent.futures libraries and go to http://localhost

I am running a function for several sets of iterables, returning a list of all results as soon as all processes are finished. def fct(variable1, variable2): # do an operation that does not necessarily take the same amount of # time for different input variables and yields result1 and result2 return result1, result2 variables1 = [1,2,3,4] variables2 = [7,8,9,0] with ThreadPoolExecutor(max_workers = 8) as executor: future = executor.map(fct,variables1,variables2) print '[%s]' % ', '.join(map(str, future)) >>> [ (12,3) , (13,4) , (14,5) , (15,6) ] How can I print intermediary results e.g. for

I am implementing futures in Pharo. I came across this website http://onsmalltalk.com/smalltalk-concurrency-playing-with-futures . I am following this example and trying to replicate it on Pharo. However, I get to this point the last step and I have no idea what ">>" means: This symbol is not also included as part of Smalltalk syntax in http://rigaux.org/language-study/syntax-across-languages-per-language/Smalltalk.html . BlockClosure>>future ^ SFuture new value: self fixTemps I can see future is not a variable or a method implemented by BlockClosure. What should I do with this part of the

I am executing a FORTRAN exe from Python. The FORTRAN exe takes many minutes to complete; therefore, I need a callback to be fired when the exe finishes. The exe does not return anything back to Python, but in the callback function I will use Python to parse output text files from the FORTRAN. For this I am using concurrent.futures and add_done_callback() , and it works. But this part of a web service and I need to have the Python method that calls subprocess.call() / Popen() to return once the FORTRAN exe is executed. Then when the FORTRAN is complete the callback function is called. def

I'm performing analyses of time-series of simulations. Basically, it's doing the same tasks for every time steps. As there is a very high number of time steps, and as the analyze of each of them is independant, I wanted to create a function that can multiprocess another function. The latter will have arguments, and return a result. Using a shared dictionnary and the lib concurrent.futures, I managed to write this : import concurrent.futures as Cfut def multiprocess_loop_grouped(function, param_list, group_size, Nworkers, *args): # function : function that is running in parallel # param_list :

The two classes represent excellent abstractions for concurrent programming, so it's a bit disconcerting that they don't support the same API. Specifically, according to the docs : asyncio.Future is almost compatible with concurrent.futures.Future . Differences: result() and exception() do not take a timeout argument and raise an exception when the future isn’t done yet. Callbacks registered with add_done_callback() are always called via the event loop's call_soon_threadsafe() . This class is not compatible with the wait() and as_completed() functions in the concurrent.futures package. The

Using the below example, how can future2 use the result of future1 once future1 is complete (without blocking future3 from being submitted)? from concurrent.futures import ProcessPoolExecutor import time def wait(seconds): time.sleep(seconds) return seconds pool = ProcessPoolExecutor() s = time.time() future1 = pool.submit(wait, 5) future2 = pool.submit(wait, future1.result()) future3 = pool.submit(wait, 10) time_taken = time.time() - s print(time_taken) This is achievable by carefully crafting a callback to submit the second operation after the first one has completed. Sadly, it is not

I've got a live concurrent.futures.ThreadPoolExecutor . I want to check its status. I want to know how many threads there are, how many are handling tasks and which tasks, how many are free, and which tasks are in the queue. How can I find out these things? There is some visibility into the Pool, and the pending workitem queue. To find out what's available, print poolx.__dict__ to see the structure. Read the ThreadPool code, it's pretty good: concurrent.futures.thread The following creates a pool with one thread. It then creates two jobs: one sleeps for 3 seconds, the other immediately returns