It seems to be impossible to make a cached thread pool with a limit to the number of threads that it can create.
Here is how static Executors.newCachedThreadPool is
You can use ThreadPoolExecutor as suggested by @sjlee
You can control the size of the pool dynamically. Have a look at this question for more details :
Dynamic Thread Pool
OR
You can use newWorkStealingPool API, which has been introduced with java 8.
public static ExecutorService newWorkStealingPool()
Creates a work-stealing thread pool using all available processors as its target parallelism level.
By default, the parallelism level is set to number of CPU cores in your server. If you have 4 core CPU server, thread pool size would be 4. This API returns ForkJoinPool type of ExecutorService and allow work stealing of idle threads by stealing tasks from busy threads in ForkJoinPool.