Nested ArrayList.ParallelStream() in custom ForkJoinPool uses threads unevenly [duplicate]

问题

I want to use my custom ForkJoinPool to have more parallelism with ArrayList.parallelStream() (by default it uses common pool).

I do this:

List<String> activities = new ArrayList<>();

for (int i = 0; i < 3000; i++) {
    activities.add(String.valueOf(i));
}

ForkJoinPool pool = new ForkJoinPool(10);
pool.submit(() ->
        activities.parallelStream()
        .map(s -> {
            try {
                System.out.println("Start task = " + s);
                Thread.sleep(100);
                System.out.println("End task = " + s);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
            return s;
        })
        .collect(toList())
).get();

and when I observe how in works in VisualVM, I see: VisualVM with parallelism 10

At some point some threads are parked and the other threads do the rest of the work. When I looked at the dump, I saw that threads, which does nothing, was in parking state.

Experimentally, it was revealed that if you create a ForkJoinPool with parallelism parameter what is a power of two, everything is OK...

List<String> activities = new ArrayList<>();

    for (int i = 0; i < 3000; i++) {
        activities.add(String.valueOf(i));
    }

    ForkJoinPool pool = new ForkJoinPool(16);
    pool.submit(() ->
            activities.parallelStream()
            .map(s -> {
                try {
                    System.out.println("Start task = " + s);
                    Thread.sleep(100);
                    System.out.println("End task = " + s);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
                return s;
            })
            .collect(toList())
    ).get();

VisualVM with parallelism 8

It can be any of powers of two less than 64 (I don't know why but it's not more than 34 threads of the ForkJoinPool), but if it is not power of two, we get the strange behaviour.

Why it happens? How to work with it?

回答1:

Instead of instanciating your own ForkJoinPool, you should invoque the factory method ForJoinPool.commonPool() which should be adapted with your available cores.

Can you give a try.

Anyway, I think you'd better have to use Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors()) instead.

来源：https://stackoverflow.com/questions/49068119/nested-arraylist-parallelstream-in-custom-forkjoinpool-uses-threads-unevenly

标签

java