Out of Memory when reading a string from SqlDataReader

Question

I\'m running into the strangest thing that I can\'t figure out. I have a SQL table with a bunch of reports stored in an ntext field. When I copied and pasted the value of on

Answer 1

Fundamentally, a System.OutOfMemoryException doesn't just occur when you are out of memory, but when you cannot allocate a single contiguous block of memory for an object. You'll often see that error when trying to create a very large array, or load a large bitmap object, or sometimes when creating large XmlDocuments...

Array and String typically need to be allocated contiguously, ie can't be broken up into pieces and allocated into empty spaces in memory.

This likely isn't a SQL issue and is more an issue with the SqlReader trying to allocate a string large enough to contain the data in a row.

You mentioned that it worked properly after a reboot, so let's assume your code is fundamentally correct (possibly can still be optimised to rather expose the data as a stream instead of buffering the recordset) and that the current symptom is environmental. A freshly rebooted machine possibly doesn't have as much fragmented memory, but as you used it more, the memory fragmented and the error returned...

You may be able to prove the contiguous memory theory by closing as many other programs as possible, and adding code to force a GC.Collect(GC.MaxGeneration) (reference) before the code with the error. This isn't a guarantee, as the memory allocated to your process may still be fragmented.

I think streaming the value might be the way to stop the error occurring, and better to avoid trying to buffer everything into a string. The downside to this is that you will keep the database connection open while the result is streamed / consumed by the rest of the program and that will bring its own overheads. I'm not sure what your code needs to do with the result, but if it needs to work with a String instance, you may need to expand the memory available to the process (several ways to help that, but may be off-topic - leave a comment and I can add to this answer if needed)