RXJS while loop for pagination

Question

I try to query all people data from the swap api. The URL swapi.co/api/people returns an object with an people array and the URL(swapi.co/api/people/?page=2) where I get the next data from. What I want to do is that the subscribe method updates the angular component every time a new page is available.

I’m new to the reactive programming model. How can I implement a while loop or a sequence of Observables?

This works for the first page:

getAllPeople(): Observable<Person[]> {
    let nextUrl = http://swapi.co/api;

    let source = Observable.create(observer => {

            this.http.get(nextUrl, { headers: this.headers })
            .map(response => {
                    let body = response.json();
                    nextUrl = body.next;
                    return nextUrl != null ? body.results as Person[] : null;
                }
            )
            .retry(5)//send same request based on the url of the previous request till the field next is null
            .catch(error => observer.error(error))
            .subscribe(persons => {
                    if (persons !== null) {
                        observer.next(persons)
                    }
                }
            );

        //observer.complete();
    })


    return source;
}

Edit: I added the retry() method. I hope it is easier to understand what I intend to do. After the first request, it sends more requests to the same URL. Unfortunately, it doesn't use the new URL I get from the previous request.

http://swapi.co/api/?page=1 -> http://swapi.co/api/?page=2 -> http://swapi.co/api/?page=3 -> null

I don’t know how many requests I need to retrieve all data.

Answer 1

By using the expand operator you can create a new Observable sequence for each value that is returned. To stop the expand from creating any new sequences, once we get a next URL that is null, we can return an empty Observable to stop expanding our sequences. Finally, we can use a reduce operator to aggregate the different pages of data into one array that we then emit back to the original subscriber. This can be accomplished as follows:

public getAllPeople(): Observable<Person[]> {
    return Observable.create(observer => {
          this.getPage("http://swapi.co/api/people/")
            .expand((data, i) => {
                return data.next ? this.getPage(data.next) : Observable.empty();
            })
            .reduce((acc, data) => {
                return acc.concat(data.results);
              }, [])
            .catch(error => observer.error(error))
            .subscribe((people) => {
                  observer.next(people);
                  observer.complete();
            });
    });
}

private getPage(url: string): Observable<{next: string, results: Person[]}> {
  return this.http.get(url, { headers: this.headers })
            .map(response => {
                    let body = response.json();
                    return {
                      next: body.next,
                      results: body.results as Person[]
                    };
                }
            );
}

Demo

Answer 2

Use expand operator like:

function getData(id) {
   let next = id<5 ? id+1 : null;
   let obj = {id:id, next:next}
   return next ?   Rx.Observable.of(obj) : Rx.Observable.empty();
 }

 getData(0)
   .expand(el=> getData(el.next))
   .subscribe(x=>console.log(x));