How to parse a text file in C# and be io bound?

Question

It is known that if you read data from disc you are IO bound and you can process/parse the read data much faster than you can read it from disc.

But this common wis

Answer 1

Throwing a profiler at it, it looks like the majority of the time is indeed spent parsing the file.

I tried your sample passing in a MemoryStream with the already read byte[] instead of a path to the ParseLines method, and the difference between parsing from a file path and parsing from in memory bytes was negligible.

In otherwords it's the processing that's done, not the reading that took up a significant amount of time.

I threw this profiler at it: http://code.google.com/p/slimtune/

And from there I can see that of the ParseLines method when called on a memory stream had the following timings:

25.34% in System.Io.StreamReader.ReadLine()
23.86% in System.Double.Parse
21.72% in System.Number.ParseInt32
20.91% in System.String.Split
--Some other not very significant methods--

So what this tells me is that even reading lines from a memory stream is somewhat slow, as are most string operations.