Improve performance of reading event log [duplicate]

Question

This question already has an answer here:

• What is the Fastest way to read event log on remote machine? 8 answers

I am queries event log of different domain controllers, I have to keep querying that after some time interval.

Below is the code I am using to query it.

public static void FindAllLog(string machineName)
        {
            //EventLog log = new EventLog("", "");
            //log.
            EventLog[] remoteEventLogs;
            // Gets logs on the local computer, gives remote computer name to get the logs on the remote computer.
            remoteEventLogs = EventLog.GetEventLogs(machineName);
            Console.WriteLine("Number of logs on computer: " + remoteEventLogs.Length);

            for (int i = 0; i < remoteEventLogs.Length; i++)
            {
                Console.WriteLine("Log : " + remoteEventLogs[i].Log);
                ReadEventLog(machineName, remoteEventLogs[i].Log, DateTime.Now.AddDays(-30));
                //ReadAppEventLog(machineName, remoteEventLogs[i].Log);                
            }
        }

public static void ReadEventLog(string machine, string logType,DateTime fromDate)
        {
            EventLog ev = new EventLog(logType, machine);
            var entry = (from EventLogEntry e in ev.Entries
                         where e.TimeGenerated >= fromDate
                         orderby e.TimeGenerated
                         select e);//.LastOrDefault();
            foreach (EventLogEntry CurrentEntry in entry)
            {
                Console.WriteLine("Event ID : " + CurrentEntry.EventID);
                Console.WriteLine("Event Source : " + CurrentEntry.Source);
                Console.WriteLine("Event TimeGenerated : " + CurrentEntry.TimeGenerated);
                Console.WriteLine("Event TimeWritten : " + CurrentEntry.TimeWritten);
                Console.WriteLine("Event MachineName : " + CurrentEntry.MachineName);
                Console.WriteLine("Entry Type : " + CurrentEntry.EntryType.ToString());
                Console.WriteLine("Message :  " + CurrentEntry.Message + "\n");
                Console.WriteLine("-----------------------------------------");
            }
        }

When first time I am querying a domain controller, I have to read log of last 30 days. Else just read latest log from the last time we left. Its taking hell lot of time to query it? I tried with WMI, same issue hell lot of time and it is sometime giving “Invalid Query Error”? How to improve this? Any model you suggest to do this task, I am doing multithreading here for each Domain Controller?

Thanks

Answer 1

To answer this question. I tried all style of reading event logs.

Using .NET2.0 approach using EventLog class, then reading using .NET3.0 approach using EventLogQuery and EventLogReader class, finally I tried WMI approach.

I have to read event logs based on time or in time slice for every 5 mins or so.

You guys will surprised to know that WMI will retrieve data way more faster then other .NETx approach and we will get more fields and no OS dependencies or firewall issues.

But other two approaches have cons.

just wanted to share this.

Thanks

Answer 2

Avoid LINQ when reading from EventLog. Try this:

// Store indices of last accessed EventLogEntries in Dictionary {logType, lastIndex}
private static readonly Dictionary<string, int> _lastIndices = new Dictionary<string, int>();

public static void FindAllLog(string machineName)
{
    //EventLog log = new EventLog("", "");
    //log.
    EventLog[] remoteEventLogs;
    // Gets logs on the local computer, gives remote computer name to get the logs on the remote computer.
    remoteEventLogs = EventLog.GetEventLogs(machineName);
    Console.WriteLine("Number of logs on computer: " + remoteEventLogs.Length);

    for (int i = 0; i < remoteEventLogs.Length; i++)
    {
        Console.WriteLine("Log : " + remoteEventLogs[i].Log);
        ReadEventLog(machineName, remoteEventLogs[i].Log, DateTime.Now.AddDays(-30));
        //ReadAppEventLog(machineName, remoteEventLogs[i].Log);                
    }
}

public static void ReadEventLog(string machine, string logType, DateTime fromDate)
{
    int lastIndex;
    EventLog ev = new EventLog(logType, machine);
    IList<EventLogEntry> entries = new List<EventLogEntry>();

    if (_lastIndices.ContainsKey(logType))
        lastIndex = _lastIndices[logType];
    else {
        lastIndex = 0;
        _lastIndices.Add(logType, 0);
    }

    // Try to avoid LINQ because it uses Enumerator and Loops EVERYTIME trough all items.
    // Start Looping from top of the list and break if Entry has Index less than lastIndex or
    // if Entry has TimeWritten less than fromDate
    for (var i = ev.Entries.Count - 1; ev.Entries[i].Index > lastIndex && ev.Entries[i].TimeWritten > fromDate; i--)
        entries.Add(ev.Entries[i]);

    if (entries.Count > 0) // Set lastIndex for corresponding logType
        _lastIndices[logType] = entries.Max(e => e.Index);

    foreach (EventLogEntry CurrentEntry in entry.OrderBy(e => e.TimeWritten))
    {
        Console.WriteLine("Event ID : " + CurrentEntry.EventID);
        Console.WriteLine("Event Source : " + CurrentEntry.Source);
        Console.WriteLine("Event TimeGenerated : " + CurrentEntry.TimeGenerated);
        Console.WriteLine("Event TimeWritten : " + CurrentEntry.TimeWritten);
        Console.WriteLine("Event MachineName : " + CurrentEntry.MachineName);
        Console.WriteLine("Entry Type : " + CurrentEntry.EntryType.ToString());
        Console.WriteLine("Message :  " + CurrentEntry.Message + "\n");
        Console.WriteLine("-----------------------------------------");
    }
}

I used here the TimeWritten property because it is more reliable than TimeGenerated. The TimeGenerated can be out of order but the TimeWritten is allways ascending as well as the Index. I hope this helps.