none
Efficient C# event log class RRS feed

  • Question

  • I see a lot of questions regarding how to read evtx files, but I can't find any resolutions that address the inherent performance issues of the EventLogReader class.  The problem is the FormatDescription() call which under the covers results in a pInvoke to native code.

    First, consider this example for reading the current log:

    System.Diagnostics.EventLog log = EventLog.GetEventLogs().FirstOrDefault(el => el.Log.Equals("Application", StringComparison.OrdinalIgnoreCase));
    foreach (EventLogEntry evt in log.Entries)
    {
        //Do stuff with log
    }

    This code can generate a data set which executes quickly, including the details of the event via the "Message" property of EventLogEntry.  On my test box I've loaded over 20,000 records in 5 seconds or less.

    Unfortunately, I can't seem to find a way to use the System.Diagnostics.EventLog class to access saved event logs, only active event logs.  To accomplish this task I've only found using the System.Diagnostics.Eventing.Reader.EventLogReader class.  Here's a sample of that:

    var query = new EventLogQuery(source, pathType, "*");
    using (var reader = new System.Diagnostics.Eventing.Reader.EventLogReader(query))
    {
        for (EventRecord eventInstance = reader.ReadEvent();
            null != eventInstance; eventInstance = reader.ReadEvent())
        {
            //Do stuff
        }
    }
    

    (sorry for mixed var/lambda, I'm trying to paraphrase my solution for clarity here)

    In this solution to get the equivalent of System.Diagnostics.EventLogEntry.Message we have to use System.Diagnostics.Eventing.Reader.EventRecord.FormatDescription(), which is where the aforementioned pInvoke comes in.  So to load the same 20,000 records the previous code loaded in say 5 seconds now takes minutes from the same machine, assuming within "do stuff" the FormatDescription() method is called.

    I've troubleshot this same exact problem in other code I worked in to the poor performance of thousands of pInvoke calls being made to transition from managed to native code.  In my past project I resorted to using native code to create a text file and parsing it.  I could do that here using wevetutil to convert EVTX to XML then parse it, but I'm hoping someone has a better solution for reading EVTX and avoiding pInvoke calls for performance.

    One thing on the requirements, a delayed call to FormatDescription() is not a solution to my problem.  The software I'm updating is intended to provide complex filtering capabilities for event logs.  The ~20k record file I've been mentioning is actually a pretty small file given what I typically analyze.  Depending on the situation, a 100MB evtx file would not be out of the ordinary.

    This is my one hurdle preventing me from moving forward on a WPF MVVM solution that I'm fairly proud of, short of some work on my data grid filters to support the complex filtering I'm aiming for.

    Thursday, October 17, 2013 2:19 AM

All replies

  • Hi Chad,

    Welcome to MSDN forums.

    According to your description, you want to find a better solution for reading Windows Event log.

    First, I would suggest you have a look at the blog via the link below.

    http://blog-mstechnology.blogspot.in/2009/08/filter-eventlog-entries-thru-c-code.html

    http://asadyousufi.blogspot.in/2010/03/read-eventlog-using-c-and-linq.html

    http://phejndorf.wordpress.com/2011/03/31/using-c-and-linq-to-read-a-windows-eventlog-file-evtx/

    At the same time, there are some suggestions in the threads with the same requirement below.

    http://stackoverflow.com/questions/13620635/enhance-reading-speed-from-event-log

    Hope is can help you.

    Regards,

    Damon


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Thursday, October 17, 2013 7:50 AM
  • Thanks for the response.  Some of the blogs are blocked at work, so I'll have to review them at home.  However, I thought I would provide feedback on the links that are accessible for me:

    First, the stack overflow link:

    As I had intended to convey, using EventLogReader and EventRecord do offer great performance but only if you do not call FormatDescription().  However, FormatDescription is the only way I have found with these classes to obtain the details of the event, which is very important for my goal.  It might take me a while to find it again, but definitely in some C# event log questions on StackOverflow commenters have made the same observations I am making.

    And the wordpress link:

    This one again is the same story since the same objects are being used, the only real difference is the yield statement.  I don't think yield applies to my requirements as I need all of my records loaded for filtering purposes up front.  Consider this snippet based upon the example:

                System.Diagnostics.Stopwatch watch = new System.Diagnostics.Stopwatch();
                watch.Reset();
                watch.Start();
                int count = 0;
                foreach (EventRecord er in records)
                {
                    count++;
                    er.FormatDescription();
                }
                watch.Stop();
                MessageBox.Show(string.Format("{0} records loaded in {1} seconds.", count, watch.ElapsedMilliseconds / 1000));

    If I comment out the er.FormatDescription() line, then I have 25,000 records that are read in 0 seconds.  However, with er.FormatDescription() in place it takes several minutes to read the same 25,000 records.  I'd give an exact time, but honestly I wrote up this entire post while waiting on the iteration to complete so I gave up waiting.

    The other two links are blocked, but the titles seem familiar to me and I expect I'll have the same response.

    Oh, to address one other suggestion: LogParser.  LogParser ends up yielding the same performance issue.  When you use the LogParser interop DLL for a C# application, the initial query simply returns an enumeration.  Iterating that enumeration to get each record of the result again costs a pInvoke, so for a large data set you see the same exact performance degradation I'm discussing with the EventRecord class, but is not present in the EventLogEntry class.


    Hopefully this helps to clarify the specific point I wanted to discuss.

    Thursday, October 17, 2013 5:17 PM
  • I found a few minutes to mess with this and created a sample console app to remove all GUI elements from the picture.  I'm focused on this sample block:

    System.Diagnostics.Stopwatch watch = new System.Diagnostics.Stopwatch();
    watch.Reset();
    watch.Start();
    int count = 0;
    
    var query = new EventLogQuery("C:\\Temp\\appprox2.evtx", PathType.FilePath, "*");
    using (var reader = new System.Diagnostics.Eventing.Reader.EventLogReader(query))
    {
        for (EventRecord eventInstance = reader.ReadEvent();
            null != eventInstance; eventInstance = reader.ReadEvent())
        {
            eventInstance.FormatDescription();
            if (count % 1000 == 0)
            {
                Console.WriteLine(string.Format("{0} records loaded at {1} ms.", count, watch.ElapsedMilliseconds));
            }
            count++;
        }
    }
    watch.Stop();
    Console.WriteLine(string.Format("{0} records loaded in {1} ms. (complete)", count, watch.ElapsedMilliseconds)); 
    Console.ReadKey();

    So again to start with, I'll comment out the call to FormatDescription and run the program.  The final output (for brevity) is:

           25525 records loaded in 541 ms. (complete)

    If I run this with the FormatDescription call we get :

           25525 records loaded in 285305 ms. (complete)

    And this is the heart of my question.  Since FormatDescription uses a native pInvoke (and registry access if you want to go that deep) it is always going to be very slow on a large data set.  What other options exist within C# for saved records?  Or is there a process by which I can load a saved log to the system, then treat it as an active log?  Windows 7 supports custom event logs, and I'm fine with a solution that requires Windows 7 or later OS.


    --Chad

    Thursday, October 17, 2013 8:31 PM
  • Hi Chad,

    In this case,I would suggest you post it in the VC# forum to find a better solution.

    http://social.msdn.microsoft.com/Forums/en-US/home?forum=csharpgeneral


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.


    Friday, October 18, 2013 6:55 AM