Java Executor with throttling/throughput control

谁都会走 提交于 2019-12-29 03:08:20
问题 I'm looking for a Java Executor that allows me to specify throttling/throughput/pacing limitations, for example, no more than say 100 tasks can be processed in a second -- if more tasks get submitted they should get queued and executed later. The main purpose of this is to avoid running into limits when hitting foreign APIs or servers. I'm wondering whether either base Java (which I doubt, because I checked) or somewhere else reliable (e.g. Apache Commons) provides this, or if I have to write

ThreadPool / Executor on both Runnable and Callable - that can exits automatically java8

拈花ヽ惹草 提交于 2019-12-24 02:09:34
问题 I am trying to use a ThreadPoolExecutor/ExecutorService in my application - it is a static global object. I use: Executors.newScheduledThreadPool(corePoolSize) - but I am having problems with shutting down the executorService. If I don't call shutdown() + awaitTermination() - then my application won't finish -even if all threads are completed. My application has threads being created by other threads - so I can't put a shutdown() anywhere in the code without blocking further threads to be run

ScheduledExecutorService, how to stop action without stopping executor?

心已入冬 提交于 2019-12-17 20:17:54
问题 I have this code: ScheduledExecutorService scheduledExecutor; ..... ScheduledFuture<?> result = scheduledExecutor.scheduleWithFixedDelay( new SomethingDoer(),0, measurmentPeriodMillis, TimeUnit.MILLISECONDS); After some event I should stop action, which Declared in run() method of the SomethingDoer , which implements Runnable . How can I do this? I can't shutdown executor, I should only revoke my periodic task. Can I use result.get() for this? And if I can, please tell me how it will work.


☆樱花仙子☆ 提交于 2019-12-15 11:12:57
【推荐】2019 Java 开发者跳槽指南.pdf(吐血整理) >>> 我之前一篇文章谈到了ThreadPoolExecutor的作用( http://my.oschina.net/xionghui/blog/494004 ),这篇文章介绍下它的原理,并根据原理分析下它的实现源码。 我们先来查看一下ThreadPoolExecutor API,看看它能实现什么功能,然后看看它是怎么实现这些功能的。 ThreadPoolExecutor API ThreadPoolExecutor API比较长,这里列出几个关键点: 核心和最大池大小:如果运行的线程少于 corePoolSize,则创建新线程来处理请求(即一个Runnable实例),即使其它线程是空闲的。如果运行的线程多于 corePoolSize 而少于 maximumPoolSize,则仅当队列满时才创建新线程。 保持活动时间:如果池中当前有多于 corePoolSize 的线程,则这些多出的线程在空闲时间超过 keepAliveTime 时将会终止。 排队: 如果运行的线程等于或多于 corePoolSize,则 Executor 始终首选将请求加入队列BlockingQueue,而不添加新的线程。 被拒绝的任务:当 Executor 已经关闭,或者队列已满且 线程数量达到 maximumPoolSize时(即线程池饱和了)

How to access the underlying queue of a ThreadpoolExecutor in a thread safe way

我的未来我决定 提交于 2019-12-12 01:06:20
问题 The getQueue() method provides access to the underlying blocking queue in the ThreadPoolExecutor, but this does not seem to be safe. A traversal over the queue returned by this function might miss updates made to the queue by the ThreadPoolExecutor. "Method getQueue() allows access to the work queue for purposes of monitoring and debugging. Use of this method for any other purpose is strongly discouraged." What would you do if you wanted to traverse the workQueue used by the

How to know when a thread stops or is stopped?

我是研究僧i 提交于 2019-12-11 12:34:21
问题 I have the next code: Executor exe = Executors.newFixedThreadPool(20); while (true) { try { exe.execute(new DispatcherThread(serverSocket.accept())); continue; } catch (SocketException sExcp) { System.exit(-1); } catch (Exception excp) { System.exit(-1); } } For each DispatcherThread I create a connection to the database (it means I have 20 connections), what I need to know is how I can close the connection to the database when the thread is stopped or it stops or finishes its flow. 回答1: You

ScheduledThreadPoolExecutors and custom queue

痴心易碎 提交于 2019-12-10 15:20:32
问题 If I use a ThreadPoolExecutor I have a variety of constructors and I can pass/use my own queue for the pool's work queue. Now I see that a ScheduledThreadPoolExecutor is a subclass of ThreadPoolExecutor but the constructors are much less. Is there a way to use a ScheduledThreadPoolExecutor and still use my own work queue? 回答1: You can extend ScheduledThreadPoolExecutor class and use a different queue then the DelayedWorkQueue that is bound to the current ScheduledThreadPoolExecutor

How to correctly use ScheduledExecutorService?

好久不见. 提交于 2019-12-10 10:36:49
问题 So this is my first time using ScheduledFuture and I admit I'm probably way over my head here. I cannot seem to get the below sample to work. Goal is simply to take two sets of actions each with it's own timeout before moving on to the next set and repeating indefinitely. static ScheduledExecutorService executor = Executors.newScheduledThreadPool(2); ScheduledFuture<?> warm = executor.scheduleWithFixedDelay(() -> { System.out.println("warmbeans"); //do more stuff here }, 0, 1000, TimeUnit

Abort countDownLatch.await() after time out

江枫思渺然 提交于 2019-12-09 03:09:12
问题 I am using an ExecutorService to implement a 3-thread pool, and CountDownLatch to monitor the completion of all threads, for further processing. ExecutorService threadExecutor = Executors.newFixedThreadPool(3); CountDownLatch countDownLatch = new CountDownLatch(3); AuthorisationHistoryTask task1 = new AuthorisationHistoryTask(commonDataThread, countDownLatch ); PreAuthHistoryTask task2 = new PreAuthHistoryTask(userID,sessionID, commonDataThread, countDownLatch ); SettlementHistTask task4 =

How to read/write a hive table from within the spark executors

杀马特。学长 韩版系。学妹 提交于 2019-12-08 11:34:58
问题 I have a requirement wherein I am using DStream to retrieve the messages from Kafka. Now after getting message or RDD now i use a map operation to process the messages independently on the executors. The one challenge I am facing is i need to read/write to a hive table from within the executors and for this i need access to SQLContext. But as far as i know SparkSession is available at driver side only and should not be used within the executors. Now without the spark session (in spark 2.1.1)