Remove Duplicate Lines From Text File?

Question

Given an input file of text lines, I want duplicate lines to be identified and removed. Please show a simple snippet of C# that accomplishes this.

Answer 1

For a long file (and non consecutive duplications) I'd copy the files line by line building a hash // position lookup table as I went.

As each line is copied check for the hashed value, if there is a collision double check that the line is the same and move to the next. (

Only worth it for fairly large files though.