Oracle Pagination strategy

Question

I want to fetch million of rows from a table between two timestamps and then do processing over it. Firing a single query and retrieving all the records at once looks to be a ba

Answer 1

You didn't say if you were planning on adjusting "X" and "Y" each time you do the pagination. If you don't then approach is probably only valid if you have a high confidence that the data is fairly static.

Consider the following example:

My table T has 100 rows date timestamp for "today", with ID=1 to 100 respectively, and I want the last 20 rows for my first page. So I do this:

select * 
from T 
where date_col = trunc(sysdate) 
order by id desc
fetch first 20 rows only

I run my query and get ID=100 down to 80. So far so good - it is all on the user's page, and they take 30 seconds mins to read the data. During that time, another 17 records have been added to the table (ID=101 to 117).

Now the user presses "Next Page"

Now I run the query again to get the next set

select * 
from T 
where date_col = trunc(sysdate) 
order by id desc
offset 20 fetch next 20 rows only

They will not see rows 80 down to 60, which would be their expectation, becuase the data has changed. They would

a) get rows ID=117 down to 97, and skip them due to the OFFSET b) then get rows ID=97 down to 77 to be displayed on screen

They'll be confused because they are seeing pretty much the same set of rows as they did on the first page.

For pagination against changing data, you generally want to stay away from the offset clause, and use your application to take note of where you got up to, ie

Page 1

select * 
from T 
where date_col = trunc(sysdate) 
order by id desc
fetch first 20 rows only

I fetch ID=100 down to 80...I take note of the 80. My next query will then be

select * 
from T 
where date_col = trunc(sysdate) 
AND ID<80
order by id desc
fetch first 20 rows only

and my next query would be

select * 
from T 
where date_col = trunc(sysdate) 
AND ID<60
order by id desc
fetch first 20 rows only

and so forth.