threadpool

I have a fixedThreadPool that I am using to run a bunch of worker threads to achieve parallel execution of a task with many components. When all threads have finished, I retrieve their results (which are quite large) using a method (getResult) and write them to a file. Ultimately, to save memory and be able to see intermediate results, I'd like each thread to write its result to the file as soon as it finishes execution and then free its memory. Ordinarily, I'd add code to that effect to the end of the run() method. However, certain other objects in this class also calls these threads, but DO

I'm trying to get to grips with pythons multiprocessing module, specifically the apply_async method of Pool . I'm trying to call a function with arguments and keyword arguments. If I call the function without kwargs it's fine but when I try to add in a keyword argument I get: TypeError: apply_async() got an unexpected keyword argument 'arg2' Below is the test code that I'm running #!/usr/bin/env python import multiprocessing from time import sleep def test(arg1, arg2=1, arg3=2): sleep(5) if __name__ == '__main__': pool = multiprocessing.Pool() for t in range(1000): pool.apply_async(test, t,

We use Retrofit/OkHttp3 for all network traffic from our Android application. So far everything seems to run quite smoothly. However, we have now occasionally had our app/process run out of file handles. Android allows for a max of 1024 file handles per process OkHttp will create a new thread for each async call Each thread created this way will (from our observation) be responsible for 3 new file handles (2 pipes and one socket). We were able to debug this exactly, where each dispached async call using .enqueue() will lead to an increase of open file handles of 3. The problem is, that the

I have a List with items that I want to download. I use a for Loop to iterate the list. For each item in this List I start a new Thread that references the item. My Problem is that I want limit the maxDownload at the same time. for (int i = downloadList.Count - 1; i >= 0; i--) { downloadItem item = downloadList[i]; if (item.Status != 1 && item.Status != 2) { ThreadStart starter = delegate { this.DownloadItem(ref item); }; Thread t = new Thread(starter); t.IsBackground = true; t.Name = item.Name; t.Priority = ThreadPriority.Normal; t.Start(); } } I read something about the ThreadPool, but then

I'm quite new to executer services. Liked doing everything myself, but I think it's time to trust these services. I want to hand by Executer a Runnable . The executer wraps that in a FutureTask and hands it back to me. Now I call poll the done() method. But I would like to be notified when then done() method would return true. There is a get() method that blocks until the Runnable has finished, but then I would need a extra thread for every job, just to see when it's finished. Can I hand my executer some extra Callable to get notified about the task finishing? What's the way to go here? I