The method publishOn of Flux doesn't not work as expected

问题

I'm trying to integrate a blocking consumer as a Flux subscriber in Reactor Aluminium-SR1. I would like to use a parallel Scheduler, to execute the blocking operations concurrently.

I've implement a main class to describe my intention:

package etienne.peiniau;

import org.reactivestreams.Subscriber;
import org.reactivestreams.Subscription;
import reactor.core.publisher.Flux;
import reactor.core.scheduler.Schedulers;
import reactor.util.function.Tuple2;

public class Main {

    public static void main(String[] args) throws InterruptedException {
        Flux.just(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20)
                .elapsed()
                .publishOn(Schedulers.parallel())
                .subscribe(new Subscriber<Tuple2<Long, Integer>>() {
                    @Override
                    public void onSubscribe(Subscription subscription) {
                        System.out.println("[" + Thread.currentThread().getName() + "] Subscription");
                        subscription.request(Long.MAX_VALUE);
                    }

                    @Override
                    public void onNext(Tuple2<Long, Integer> t2) {
                        System.out.println("[" + Thread.currentThread().getName() + "] " + t2);
                        try {
                            Thread.sleep(1000); // long operation
                        } catch (InterruptedException e) {
                            e.printStackTrace();
                        }
                    }

                    @Override
                    public void onError(Throwable throwable) {
                        System.err.println("[" + Thread.currentThread().getName() + "] Error: " + throwable.getMessage());
                    }

                    @Override
                    public void onComplete() {
                        System.out.println("[" + Thread.currentThread().getName() + "] Complete");
                    }
                });
        // Waiting for the program to complete
        System.out.println("[" + Thread.currentThread().getName() + "] Main");
        Thread.sleep(100000);
    }

}

The output of this code is the following:

[main] Subscription
[main] Main
[parallel-1] [3,1]
[parallel-1] [1000,2]
[parallel-1] [1001,3]
[parallel-1] [1000,4]
[parallel-1] [1000,5]
[parallel-1] [1000,6]
[parallel-1] [1001,7]
[parallel-1] [1000,8]
[parallel-1] [1000,9]
[parallel-1] [1000,10]
[parallel-1] [1000,11]
[parallel-1] [1001,12]
[parallel-1] [1000,13]
[parallel-1] [1000,14]
[parallel-1] [1000,15]
[parallel-1] [1000,16]
[parallel-1] [1001,17]
[parallel-1] [1000,18]
[parallel-1] [1000,19]
[parallel-1] [1000,20]
[parallel-1] Complete

My problem is that the long operation is always executed on the thread parallel-1 and every 1 second.

I've tried to increase parallelism manually or to use an elastic Scheduler, but the result is the same.

I was thinking that publishOn method was specially designed for this use case. Can you tell me if I misunderstood something ?

回答1:

Actually it works as expected you can see that all values where processed in parallel - elapsed time is nearly the same, but you always receive elements within the same thread and with that way each time you wait 1 second.

I guess that in simple Flux parallel doesn't mean more thread, it means to do work in parallel. If you for example run code like:

Flux.fromIterable(IntStream.range(0, 20).boxed().collect(Collectors.toList()))
            .map(i -> {
                System.out.println("map [" + Thread.currentThread().getName() + "] " + i);
                return i;
            })
            .elapsed()
            .publishOn(Schedulers.single())
            .subscribeOn(Schedulers.single())
            .subscribe(t2 -> {
                System.out.println("subscribe [" + Thread.currentThread().getName() + "] " + t2);
            });

You will see results:

map [single-1] 0
map [single-1] 1
...
subscribe [single-1] [4,0]
subscribe [single-1] [0,1]
...

And you can see that first it does map for all elements and then consume. If you change publishOn to .publishOn(Schedulers.parallel()) you will see:

map [single-1] 3
subscribe [parallel-1] [5,0]
map [single-1] 4
subscribe [parallel-1] [0,1]
map [single-1] 5
...

Now it does both operations in parallel threads at once. I'm not sure that I understand everything correctly.

There is specific ParallelFlux for parallel execution. In example below everything will be done on different threads:

Flux.fromIterable(IntStream.range(0, 20).boxed().collect(Collectors.toList()))
        .elapsed()
        .parallel()
        .runOn(Schedulers.parallel())
        .subscribe(t2 -> {
            System.out.println("[" + Thread.currentThread().getName() + "] " + t2);
            try {
                Thread.sleep(1000); // long operation
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }, throwable -> {
            System.err.println("[" + Thread.currentThread().getName() + "] Error: " + throwable.getMessage());
        }, () -> {
            System.out.println("[" + Thread.currentThread().getName() + "] Complete");
        }, subscription -> {
            System.out.println("[" + Thread.currentThread().getName() + "] Subscription");
            subscription.request(Long.MAX_VALUE);
        });

Result looks like this:

[parallel-1] [8,0]
[parallel-2] [0,1]
[parallel-3] [0,2]
[parallel-4] [0,3]
[parallel-1] [0,4]
...

So it uses few threads to process results. And it's truly parallel in my point of view.

Also note that if you use method .subscribe(Subscriber<? super T> s) all results will be consumed in sequential way and to consume them in parallel you should use:

public void subscribe(Consumer<? super T> onNext, Consumer<? super Throwable>
            onError, Runnable onComplete, Consumer<? super Subscription> onSubscribe)

or any other overloaded method with Consumer<? super T> onNext,... arguments

来源：https://stackoverflow.com/questions/44567381/the-method-publishon-of-flux-doesnt-not-work-as-expected