MySQL : retrieve a large select by chunks

Question

I have select with more then

70 milion rows

I\'d like to save the selected data into the one large csv file on win2012 R2

Answer 1

The LIMIT OFFSET approach slows query down when a size of the data is very large. Another approach is to use something called Keyset pagination. It requires a unique id in your query, which you can use as a bookmark to point to the last row of the previous page. The next page is fetched using the last bookmark. For instance:

SELECT user_id, name, date_created
FROM users
WHERE user_id > 0
ORDER BY user_id ASC
LIMIT 10 000;

If the resultset above returns the last row with user_id as 12345, you can use it to fetch the next page as follows:

SELECT user_id, name, date_created
FROM users
WHERE user_id > 12345
ORDER BY user_id ASC
LIMIT 10 000;

For more details, you may take a look at this page.