It seems to be impossible to make a cached thread pool with a limit to the number of threads that it can create.
Here is how static Executors.newCachedThreadPool is
The problem was summarized as follows:
I want something exactly like the cached thread pool (it creates threads on demand and then kills them after some timeout) but with a limit on the number of threads that it can create and the ability to continue to queue additional tasks once it has hit its thread limit.
Before pointing to the solution I will explain why the following solutions don't work:
new ThreadPoolExecutor(0, 3, 60L, TimeUnit.SECONDS, new SynchronousQueue<>());
This will not queue any tasks when the limit of 3 is reached because SynchronousQueue, by definition, cannot hold any elements.
new ThreadPoolExecutor(0, 3, 60L, TimeUnit.SECONDS, new LinkedBlockingQueue<>());
This will not create more than a single thread because ThreadPoolExecutor only creates threads exceeding the corePoolSize if the queue is full. But LinkedBlockingQueue is never full.
ThreadPoolExecutor executor = new ThreadPoolExecutor(3, 3, 60, TimeUnit.SECONDS,
new LinkedBlockingQueue());
executor.allowCoreThreadTimeOut(true);
This will not reuse threads until the corePoolSize has been reached because ThreadPoolExecutor increases the number of threads until the corePoolSize is reached even if existing threads are idle. If you can live with this disadvantage then this is the easiest solution to the problem. It is also the solution described in "Java Concurrency in Practice" (footnote on p172).
The only complete solution to the described problem seems to be the one involving overriding the queue's offer method and writing a RejectedExecutionHandler as explained in the answers to this question: How to get the ThreadPoolExecutor to increase threads to max before queueing?