Efficient way to delete a line from a text file

Question

I need to delete a certain line from a text file. What is the most efficient way of doing this? File can be potentially large(over million records).

UPDATE: below is

Answer 1

In my blog, I have benchmarked various I/O methods from C# in order to determine the most efficient way of doing file I/O. In general, you are better off using the Windows ReadFile and WriteFile functions. The next fastest way to read files in is through FileStream. To get good performance, read the files in blocks at a time instead of a line at a time and then do your own parsing. The code that you can download from my blog gives you an example on how to do this. There is also a C# class that encapsulates the Windows ReadFile / WriteFile functionality and is quite easy to use. See my blog for details at:

http://designingefficientsoftware.wordpress.com/2011/03/03/efficient-file-io-from-csharp

Bob Bryan MCSD

Answer 2

If you absolutely have to use a text file and cannot switch to a database, maybe you want to designate a wierd symbol at the beginning of a line to mean "line deleted". Just have your parser ignore those lines, like comment lines in config files etc.

Then have a periodic "compact" routine like Outlook, and most database systems do, which re-writes the entire file excluding the deleted lines.

I would strongly go with Think Before Coding's answer recommending a database or other structured file.