Need a way to sort a 100 GB log file by date [closed]

Answer 1

Pre-emptive comment: My answer only addresses the sub-problem of parsing date time values.

DateTime.Parse contains checks for all possible date formats. If you have a fix format you can optimize parsing quite well. A simple optimization would be to convert the characters directly:

class DateParserYyyyMmDd
{
    static void Main(string[] args)
    {
        string data = "2010-04-22";

        DateTime date = Parse(data);
    }

    struct Date
    {
        public int year;
        public int month;
        public int day;
    }

    static Date MyDate;

    static DateTime Parse2(string data)
    {
        MyDate.year = (data[0] - '0') * 1000 + (data[1] - '0') * 100 
            + (data[2] - '0') * 10 + (data[3] - '0');
        MyDate.month = (data[5] - '0') * 10 + (data[6] - '0');
        MyDate.day = (data[8] - '0') * 10 + (data[9] - '0');

        return new DateTime(MyDate.year, MyDate.month, MyDate.day);
    }
}