Java ExecutorService heap space problems

问题

I hava a Java mulithreading question. I have the following worker class:

public class ThreadWorker implements Runnable {

    //some code in here

    public void run(){
      // invokes some recursion method in the ThreadWorker itself,
      // which will stop eventually
    {
}

To work with threads I'm using an ExecutorService:

public static int THREAD_NUMBER = 4;
public static ExecutorServide es = Executors.newFixedThreadPool(THREAD_NUMBER);

Adding instances of ThreadWroker class happens here:

public void recursiveMethod(Arraylist<Integers> elements, MyClass data){
     if (elements.size() == 0 && data.qualifies()){
         ThreadWorker tw = new ThreadWorker(data);
         es.execute(tw);
         return;
     }



     for (int i=0; i< elements.size(); i++){
          // some code to prevent my problem
          MyClass data1 = new MyClass(data);
          MyClass data2 = new MyClass(data); 
          ArrayList<Integer> newElements = (ArrayList<Integer>)elements.clone();
          data1.update(elements.get(i));
          data2.update(-1 * elements.get(i));
          newElements.remove(i);
          recursiveMethod(newElements, data1);
          recursiveMethod(newElements, data2);     
     {    
}

The problem is that the depth of the recursion tree is quite big, so as it's width, so a lot of ThreadWorkers are added to the ExecutorService, so after some time on the big input a get

Exception in thread "pool-1-thread-2" java.lang.OutOfMemoryError: Java heap space

which is caused, as I think because of a ginormous number of ThreadWorkers i'm adding to ExecutorSirvice to be executed, so it runs out of memory. Every ThreadWorker takes about 40 Mb of RAM for all it needs.

Is there a method to get how many threads (instances of classes implementing runnable interface) have been added to ExecutorService? So I can add it in the shown above code (int the " // some code to prevent my problem"), as

while ("number of threads in the ExecutorService" > 10){
    Thread.sleep(10000);
}

so I won't go to deep or to broad with my recursion and prevent those exception-throwing situations.

Sincerely, Sergey Aganezov jr.

回答1:

How about creating a ThreadPoolExecutor backed by a BlockingQueue using ThreadPoolExecutor.CallerRunsPolicy.

This way, when there are no worker threads available to run a task on, the main thread (which is adding the new jobs) runs the task itself, which prevents any more jobs from being added.

There are more details on the constructor options for ThreadPoolExecutor on its Javadoc page.

回答2:

I think your case is a good match for the "fork-join" framework of the Java JDK. (Google for that keyword.)

Fork-Join helps you to trim down the number of jobs in your queue by delaying the "split" as far as possible.

You have to reformulate your code into that philosophy though.

来源：https://stackoverflow.com/questions/10060570/java-executorservice-heap-space-problems

标签

java

multithreading

exception

executorservice