Making around 20 HTTP calls and passing data to the database using Java

问题

I have a collection of 20 items, I will create a loop for the items and make API Calls to get the data, based on the data returned, I will have to update in the database. This requirement is simple and I am able to accomplish in plain Java.

Now for performance, I am learning about using RxJava. I went through many articles in the internet and found that people refer to the async-http-client library for async http calls, I find that the library is out of date and the maintainer is planning for a hand-over to someone else, the one given in RxJava library is also like developed in 2014. Since I am new to RxJava, can you please help me with the right approach.

I am currently getting all the data and converting to observables like below

Observable<ENV> envs= Observable.fromIterable(allEnvs);

I also need to get some help like is the above code fine or should I create like the following for the observable construction, this is the snippet in groovy which I will have to write in Java.

val createObserver = Observable.create(ObservableOnSubscribe<String> { emitter ->
    emitter.onNext("Hello World")
    emitter.onComplete()
})

Kindly help me in choosing the best approach

回答1:

Imagine that the http call is represented by class below :

public class HttpCall implements Callable<String> {

    private final int i;

    private HttpCall(int i) {
        this.i = i;
    }

    @Override
    public String call() {
        try {
            Thread.sleep(2000);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }

        return "Something for : " + i;
    }
}

It waits 2 sec and then emits a string (the http call result).

To combine all the items resulting from different http calls we can use merge operator. But before that we need to transform the Callable to an Observable by using fromCallable operator.

void sequentially() {
    List<Observable<String>> httpRequests = IntStream.range(0, 20)
            .mapToObj(HttpCall::new)
            .map(Observable::fromCallable)
            .collect(Collectors.toList());

    Observable.merge(httpRequests)
            .timestamp(TimeUnit.SECONDS)
            .subscribe(e -> System.out.println("Elapsed time : " + e.time() + " -- " + e.value() + ". Executed on thread : " + Thread.currentThread().getName()));
}

Because all the requests are executed on the same thread, the order is maintained :

Elapsed time : 1602122218 -- Something for : 0. Executed on thread : main
Elapsed time : 1602122220 -- Something for : 1. Executed on thread : main
Elapsed time : 1602122222 -- Something for : 2. Executed on thread : main
...

As you can see the items are separated by 2 sec.

To run each request in its own thread we need to tell Rx that we need a thread for each call. Easy-peasy, just switch to one of the suggested schedulers. IO its what we need (as it's an IO operation).

void parallel( {
    List<Observable<String>> httpRequests = IntStream.range(0, 20)
            .mapToObj(HttpCall::new)
            .map(httpCall -> Observable.fromCallable(httpCall)
                                .subscribeOn(Schedulers.io())
            ) // take a thread from the IO pool
            .collect(Collectors.toList());

    Observable.merge(httpRequests)
            .timestamp(TimeUnit.SECONDS)
            .subscribe(e -> System.out.println("Elapsed time : " + e.time() + " -- " + e.value() + ". Executed on thread : " + Thread.currentThread().getName()));
}

This time the order is not guarenteed and they are produced at almost the same time :

Elapsed time : 1602123707 -- Something for : 2. Executed on thread : RxCachedThreadScheduler-3
Elapsed time : 1602123707 -- Something for : 0. Executed on thread : RxCachedThreadScheduler-1
Elapsed time : 1602123707 -- Something for : 1. Executed on thread : RxCachedThreadScheduler-1
...

---

The code could be shorten like :

Observable.range(0, 20)
                .map(HttpCall::new)
                .flatMap(httpCall -> Observable.fromCallable(httpCall).subscribeOn(Schedulers.io()))
                .timestamp(TimeUnit.SECONDS)
                .subscribe(e -> System.out.println("Elapsed time : " + e.time() + " -- " + e.value() + ". Executed on thread : " + Thread.currentThread().getName()));

merge uses flatMap behind scenes.

来源：https://stackoverflow.com/questions/64156263/making-around-20-http-calls-and-passing-data-to-the-database-using-java