Write to a file from multiple threads asynchronously c#

Question

Here is my situation. I would like to make writing to the file system as efficient as possible in my application. The app is multi-threaded and each thread can possibly wr

Answer 1

What I would do is have separate worker thread(s) dedicated to the task of writing files out. When one of your other threads needs to write some data out, it should call a function to add the data to an ArrayList (or some other container/class). Inside this function, there should be a lock statement at the top to prevent more than one thread from executing simultaneously. After adding the reference to the ArrayList it returns and continues on with its chores. There are a couple of ways to handle the writing thread(s). Probably the simplest is to simply put it into an infinite loop with a sleep statement at the end so that it does not chew up your cpu(s). Another way is to use thread primitives and go into a wait state when there is no more data to be written out. This method implies that you would have to activate the thread with something like the ManualResetEvent.Set method.

There are many different ways to read in and write out files in .NET. I have written a benchmark program and give the results in my blog:

http://designingefficientsoftware.wordpress.com/2011/03/03/efficient-file-io-from-csharp

I would recommend using the Windows ReadFile and WriteFile methods if you need performance. Avoid any of the asynchronous methods since my benchmark results show that you get better performance with synchronous I/O methods.