Caching a binary file in C#

Question

Is it possible to cache a binary file in .NET and do normal file operations on cached file?

Answer 1

Any modern OS has a caching system built in, so in fact whenever you interact with a file, you are interacting with an in-memory cache of the file.

Before applying custom caching, you need to ask an important question: what happens when the underlying file changes, so my cached copy becomes invalid?

You can complicate matters further if the cached copy is allowed to change, and the changes need to be saved back to the underlying file.

If the file is small, it's simpler just to use MemoryStream as suggested in another answer.

If you need to save changes back to the file, you could write a wrapper class that forwards everything on to MemoryStream, but additionally has an IsDirty property that it sets to true whenever a write operation is performed. Then you can have some management code that kicks in whenever you choose (at the end of some larger transaction?), checks for (IsDirty == true) and saves the new version to disk. This is called "lazy write" caching, as the modifications are made in memory and are not actually saved until sometime later.

If you really want to complicate matters, or you have a very large file, you could implement your own paging, where you pick a buffer size (maybe 1 MB?) and hold a small number of byte[] pages of that fixed size. This time you'd have a dirty flag for each page. You'd implement the Stream methods so they hide the details from the caller, and pull in (or discard) page buffers whenever necessary.

Finally, if you want an easier life, try:

http://www.microsoft.com/Sqlserver/2005/en/us/compact.aspx

It lets you use the same SQL engine as SQL Server but on a file, with everything happening inside your process instead of via an external RDBMS server. This will probably give you a much simpler way of querying and updating your file, and avoid the need for a lot of hand-written persistence code.