I hit an OutOfMemoryException with List - is this the limit or am I missing something?

Question

Given the opportunity to rewrite, I would, but anyway, the code as it stands:

List foobar;

Then we add a bunch of strings to

Answer 1

I've posted what I exactly did here, worth giving it a go. Again steps are:

1. On each iteration query portion of data using an stored proc
2. Transfer them

3. Move to the next portion

   List returnList;
   int index = 0;
   SqlCommand cmd = new SqlCommand("ExampleStoredProc", conn);
   cmd.CommandType = CommandType.StoredProcedure;
   while (true)
   {
       cmd.Parameters.Add(
           new SqlParameter("@index", index));
       SqlDataReader dr = cmd.ExecuteReader();
       if (dr.HasRows)
       {
           returnList = new List();
           returnList.Add(dr.GetString(0).Trim());
           //transfer data here
       }
       else
       {
           break;
       }
       index++;
   }
   

and the stored proc should be something like this:

CREATE PROCEDURE ExampleStoredProc
    @index INT
AS
BEGIN
    SELECT * 
    FROM  veryBigTable
    WHERE Id >= (@index *1000) AND Id < ((@index + 1) * 1000)
END
GO

I'll definitely work no matter how many records you have, just the more data you have, longer it'll take to finish.