“Chunked” MemoryStream

Question

I\'m looking for the implementation of MemoryStream which does not allocate memory as one big block, but rather a collection of chunks. I want to store a few GB of data in m

Answer 1

SparseMemoryStream does this in .NET it's in buried deep down in an internal class library though -- the source code is available of course, since Microsoft put it all out there as open source.

You can grab the code for it here: http://www.dotnetframework.org/default.aspx/4@0/4@0/DEVDIV_TFS/Dev10/Releases/RTMRel/wpf/src/Base/MS/Internal/IO/Packaging/SparseMemoryStream@cs/1305600/SparseMemoryStream@cs

That being said, I highly recommend not using it as is -- At the very least remove all the calls to IsolatedStorage for starters, as this seems to be the cause of no end of bugs* in the framework's packaging API.

(*: In addition to spreading the data around in streams, if it gets too big, it basically reinvents swap files for some reason -- in the user's Isolated Storage no less -- and coincidentally, most MS products that allow for .NET based add-ins do not have their app domains setup in such a way that you can access Isolated Storage -- VSTO add-ins are notorious for suffering from this issue, for example.)