concurrent.futures

问题 There are three questions as possible duplicates (but too specific): How to properly set up multiprocessing proxy objects for objects that already exist Share object with process (multiprocess) Can I use a ProcessPoolExecutor from within a Future? By answering this question all three other questions can be answered. Hopefully I make myself clear: Once I created an object in some process created by multiprocessing: How do I pass a reference to that object to an other process? (not so important

问题 I am trying to use the pool of workers provided by concurrent.futures.ProcessPoolExecutor to speed up the performance of a method inside a tkinter class. This is because executing the method is cpu intensive and "parallelizing" it should shorten the time to complete it. I hope to benchmark it's performance against a control - a serial execution of the same method. I have written a tkinter GUI test code to perform this benchmark. The serial execution of the method works but the concurrent part

问题 I'm doing some multiprocessing, and I'd like to understand why the following piece of code sometimes (but never at the first iteration of the loop) generates the following error : Exception ignored in: <bound method Image.__del__ of <tkinter.PhotoImage object at 0x7f6c40d25ba8>> Traceback (most recent call last): File "/usr/lib/python3.5/tkinter/__init__.py", line 3359, in __del__ self.tk.call('image', 'delete', self.name) RuntimeError: main thread is not in main loop Tcl_AsyncDelete: async

问题 I have a futures pool , and each future works with the same akka Actor System - some Actors in system should be global, some are used only in one future. val longFutures = for (i <- 0 until 2 ) yield Future { val p:Page = PhantomExecutor(isDebug=true) Await.result( p.open("http://www.stackoverflow.com/") ,timeout = 10.seconds) } PhantomExecutor tryes to use one shared global actor (simple increment counter) using system.actorSelection def selectActor[T <: Actor : ClassTag](system:ActorSystem

问题 Disclaimer : I'm new to Python in general. I have a small experience with Go, where implementing a queue using a channel is really easy. I want to know how can I implement a Queue with ProcessPoolExecutor in Python 3. I want my N number of process to access a single queue so that I can just insert many jobs in the queue via the main thread, then the processes will just grab the jobs in the Queue. Or if there is a better way to share a list/queue between multiple processes. (Job Queue/ Worker

问题 Disclaimer : I'm new to Python in general. I have a small experience with Go, where implementing a queue using a channel is really easy. I want to know how can I implement a Queue with ProcessPoolExecutor in Python 3. I want my N number of process to access a single queue so that I can just insert many jobs in the queue via the main thread, then the processes will just grab the jobs in the Queue. Or if there is a better way to share a list/queue between multiple processes. (Job Queue/ Worker

问题 I have the below code for creating threads and running them. from concurrent.futures import ThreadPoolExecutor import threading def task(n): result = 0 i = 0 for i in range(n): result = result + i print("I: {}".format(result)) print(f'Thread : {threading.current_thread()} executed with variable {n}') def main(): executor = ThreadPoolExecutor(max_workers=3) task1 = executor.submit(task, (10)) task2 = executor.submit(task, (100)) if __name__ == '__main__': main() When i run the code in my

问题 I have a pandas DataFrame with hundreds of thousands of rows, and I want to apply a time-consuming function on multiple columns of that DataFrame in parallel. I know how to apply the function serially. For example: import hashlib import pandas as pd df = pd.DataFrame( {'col1': range(100_000), 'col2': range(100_000, 200_000)}, columns=['col1', 'col2']) def foo(col1, col2): # This function is actually much more time consuming in real life return hashlib.md5(f'{col1}-{col2}'.encode('utf-8'))

问题 I have a program that takes in a list. For each value in this list, it retrieves another list and processes this other list. Basically, it's a 3-depth tree which need to do possibly expensive processing at each node. Each nodes needs to be able to process the results of its children. What I'd like to be able to do is to map from the inputs in the first layer list to the results of each node. In each of these processes though, I would like to map the result from the next layer down. What I'm

问题 I am executing a FORTRAN exe from Python. The FORTRAN exe takes many minutes to complete; therefore, I need a callback to be fired when the exe finishes. The exe does not return anything back to Python, but in the callback function I will use Python to parse output text files from the FORTRAN. For this I am using concurrent.futures and add_done_callback() , and it works. But this part of a web service and I need to have the Python method that calls subprocess.call() / Popen() to return once