How to provide an API client with 1,000,000 database results?

Question

Following up from my previous question:

Using "Cursors" for paging in PostgreSQL

What is a good way to provide an API client with 1,000,000 databas

Answer 1

Have the API accept an offset to start from and the number of records to return. This is a sort of paging where the client can determine how many records to return in one page request. The API should also return the total number of records possible for the query so the client knows how many "pages", or optionally it can derive when it has retrieved the last records when the number of records returned is zero or less than the number of records requested. You can control this in your PostgresSQL query by using the OFFSET clause (which record to start retrieving at) and the LIMIT clause (number of records to return) in your SELECT statement.