I have a text file that contains about 100000 articles. The structure of file is:
.Document ID 42944-YEAR:5
.Date 03\\08\\11
.Cat political
Article Content 1
.
If you are using .NET Framework 4, there is a new static method on System.IO.File called ReadLines that returns an IEnumerable of string. I believe it was added to the framework for this exact scenario; however, I have yet to use it myself.
MSDN Documentation - File.ReadLines Method (String)
Related Stack Overflow Question - Bug in the File.ReadLines(..) method of the .net framework 4.0