threadpool

Using multiprocessing for finding network paths

不羁岁月 提交于 2019-12-04 17:23:53
I am currently using the networkx function *all_simple_paths* to find all paths within a network G, for a given set of source and target nodes. On larger/denser networks this process is incredibly intensive. I would like to know if multiprocessing could conceivably be used on this problem, and if anybody had any ideas on how that might be implemented, through creating a Pool etc. import networkx as nx G = nx.complete_graph(8) sources = [1,2] targets = [5,6,7] for target in targets: for source in sources: for path in nx.all_simple_paths(G, source=source, target=target, cutoff=None): print(path)

Java ExecutorService callback on thread terminate

匆匆过客 提交于 2019-12-04 17:09:23
问题 I am using cached thread pool ExecutorService to run some asynchronous background tasks. I've provided my ThreadFactory that hands out threads to the ExecutorService (whenever it needs them). My understanding of the cached thread pool is that after the thread is sitting idle for 60 seconds it is termniated by the ExecutorService. I want to perform some state cleanup when my thread is about to be terminated. What is the best way to achieve this? The ExecutorService does not readily provide

CherryPy, Threads, and Member Variables; potential issues?

时光毁灭记忆、已成空白 提交于 2019-12-04 16:53:47
Let's say I have the following simple class: import cherrypy import os class test: test_member = 0; def __init__(self): return def index(self): self.test_member = self.test_member + 1 return str(self.test_member) index.exposed = True conf = os.path.join(os.path.dirname(__file__), 'config.ini') if __name__ == '__main__': # CherryPy always starts with app.root when trying to map request URIs # to objects, so we need to mount a request handler root. A request # to '/' will be mapped to HelloWorld().index(). cherrypy.config.update({'server.socket_host': '0.0.0.0'}) cherrypy.quickstart(test(),

how to modify ThreadPoolTaskExecutor at runtime through jmx

旧巷老猫 提交于 2019-12-04 16:49:22
I'm having trouble modifying my MBean properties through JConsole. I have a Threading bean which invoked with: public static void main(String[] args) throws Exception { // JMX new SimpleJmxAgent(); // spring executor context ApplicationContext ctx = new FileSystemXmlApplicationContext( "src/resources/ThreadContent.xml"); startThreads(ctx); } private static void startThreads(ApplicationContext ctx) { TaskExecutor tE = (TaskExecutor) ctx.getBean("TaskExecutor"); System.out.println("Starting threads"); for (int i = 0; i < 10; i++) { tE.execute(new RepeatingGrpPoC()); } ThreadContent.xml contains

Why does a ScheduledExecutorService not run a task again after an exception is thrown?

喜你入骨 提交于 2019-12-04 15:26:39
问题 For executing periodical tasks, I looked at Timer and ScheduledThreadPoolExecutor (with a single thread) and decided to use the latter, because in the reference for Executors.newSingleThreadScheduledExecutor(), it says: Note however that if this single thread terminates due to a failure during execution prior to shutdown, a new one will take its place if needed to execute subsequent tasks. My plan was to use this as a safeguard against uncaught exceptions in a watchdog piece of code that I

multiprocessing.pool.MaybeEncodingError: Error sending result: Reason: 'TypeError(“cannot serialize '_io.BufferedReader' object”,)'

孤者浪人 提交于 2019-12-04 15:01:56
I get the following error: multiprocessing.pool.MaybeEncodingError: Error sending result: '<multiprocessing.pool.ExceptionWithTraceback object at 0x7f758760d6a0>'. Reason: 'TypeError("cannot serialize '_io.BufferedReader' object",)' When running this code: from operator import itemgetter from multiprocessing import Pool import wget def f(args): print(args[1]) wget.download(args[1], "tests/" + target + '/' + str(args[0]), bar=None) if __name__ == "__main__": a = Pool(2) a.map(f, list(enumerate(urls))) #urls is a list of urls. What does the error mean and how can I fix it? First couple of

Does Thread.Sleep affect the ThreadPool?

拜拜、爱过 提交于 2019-12-04 14:51:48
I need to control one thread for my own purposes: calculating, waiting, reporting, etc... In all other cases I'm using the ThreadPool or TaskEx. In debugger, when I'm doing Thread.Sleep() , I notice that some parts of the UI are becoming less responsible. Though, without debugger seems to work fine. The question is: If I'm creating new Thread and Sleep() 'ing it, can it affect ThreadPool/Tasks? EDIT: here are code samples: One random place in my app: ThreadPool.QueueUserWorkItem((state) => { LoadImageSource(imageUri, imageSourceRef); }); Another random place in my app: var parsedResult = await

Close foreground thread gracefully on windows service stop

给你一囗甜甜゛ 提交于 2019-12-04 14:10:31
In my windows service I create one "parent" foreground thread that in turn spawns "child" threads using ThreadPool (which means they are background) to execute tasks. What is the best way to close foreground thread gracefully on windows service stop? Here is my current implementation (stripped out of task-specific logic): public partial class TaskScheduler : ServiceBase { private static AutoResetEvent _finishedTaskAutoResetEvent = new AutoResetEvent(false); //This flag is used to increase chances of the Spawning Thread to finish gracefully when service stops. private bool StopRequested { get;

Requests are queuing in Azure AppService though it has enough threads in threadpool

寵の児 提交于 2019-12-04 13:43:46
问题 I have written an api using asp.net webapi and deployed it in azure as Appservice. Name of my controller is TestController and My action method is something like bellow. [Route("Test/Get")] public string Get() { Thread.Sleep(10000); return "value"; } So for each request it should wait for 10 sec before return string "value". I have also written another endpoint to see the number of threads in threadpool working for executing requests. That action is something like bellow. [Route("Test

Lot of UDP requests lost in UDP server with Netty

ぃ、小莉子 提交于 2019-12-04 13:17:14
问题 I wrote a simple UDP Server with Netty that simply prints out in logs the messages (frames) received. To do that, I created a simple frame decoder decoder and a simple message handler. I also have a client that can send multiple requests sequentially and/or in parallel. When I configure my client tester to send for example few hundred of requests sequentially with a small delay between them, my server written with Netty receives them all properly. But at the moment I increase the number of