Getting top n to n rows from db2

Question

I need to split a huge table in to chunks.

Fetching data from DB2 and processing in SSIS

iteration 1 : Get first 10 rows and process it

Answer 1

Use OFFSET x FETCH FIRST y ROWS ONLY option to read data by chunk

To benefit from this method in SSIS, you should follow these steps:

1. Add Execute SQL Task to get the rows count and store it into a variable.
2. Add a Forloop container to loop over a range of numbers until reaching the row count
3. Inside the Forloop container, add a Data Flow Task that contains an OLEDB Source or ODBC and OLEDB Destination

4. In OLEDB/ODBC Source, Set the Access mode to SQL Command and define an epxression similar to:

   "SELECT * FROM MYTABLE ORDER BY ID_COLUMN
   OFFSET " + (DT_WSTR,50)@[User::IncrementValue] + "
   FETCH FIRST " + (DT_WSTR,50)@[User::IncrementValue] + " ROWS ONLY" 
   

The following answer is a step by step guide to load data by chunks from SQLite, you can follow it just change the SQL Command syntax as mentioned:

• Reading Huge volume of data from Sqlite to SQL Server fails at pre-execute

References

• SQL Paging With Limit And Offset In DB2 For i
• Using the OFFSET clause with a cursor
• DB2 Using LIMIT and OFFSET
• Paging through result sets using LIMIT and OFFSET or ROWNUM