none
Perview looses data when circular buffer wraps RRS feed

  • Question

  • First, unsure if this is the best choice of forum to ask a PerfView question.   If not, please direct me to a better forum to ask.

    The problem I have is when enough events are generated to cause the ETL file to wrap, the collected data is not viewable in PerfView.

    I'm testing writing events from a class derived from EventSource.   I use either Logman or PerfView to turn on data collection.

    After generating a large number of events, I issue command through Logman or Perfview to stop data collection, and a ETL file is created.

    I configure Logman or PerfView to use a 10M circular file.

    Let's say my program generates some number of events less than 10M.   For example, after stopping event collection, the ETL file is 5M.

    Viewing the event data in PerfView looks great.  No problems.

    I change the test to generate some number of events which causes the file to wrap.  (Because the test is generating more than 10M of events.)

    Because the collection file was specified to be circular, the file grows no more than 10M.

    In this scenario, when I use PerfView to view the events, the 'meta data' is gone.  

    The 'EventName" column shows only Provider{guid}/EventID(1)-- not the 'friendly name' of the eventsource.

    The last column in PerfView shows only "Datalength=xyz"-- not the data my EventSource class generated.

    Summary:  Works great if the event generator doesn't cause the file to wrap.    No useful data if enough events generated to cause the ETL file to wrap.

    Is there a setting in PerfView I'm overlooking? 

    Is there some other way to configure the ETL file to wrap to maintain data so the collected data is viewable via PerfView.

    If more information of configuration settings I'm using in Logman and a sample program to generate the events is needed, let me know.

    Thanks,

    Dan

    Thursday, May 9, 2013 3:28 AM

Answers

  • I would have expected this behavior if you collect the data with logman but not if you collected the data with PerfView.    It can also happen if the process that contains the EventSource terminates after a long time but before data collection stops.    If indeed you have a problem when the data is collected with PerfView (latest version on the web) and it is not the case above then post the repro instructions and I will fix it. 

    The basic issue is that the metadata associated with EventSource event is logged to the ETL file as just another event.  By default this event gets logged at the start of the trace.  For a circular buffer, these first events eventually get overwritten which is why it fails for Logman.    However PerfView compensates for this by including all the EventSources in the 'rundown' that it does when a trace ends.  This causes the meta-data to be re-emitted and thus present for decoding.  

    Finally PerfView does have some obscure features that can also help.   You can do PerfView userCommand DumpEventSourceManifests FILE.ETL which will create a directory called 'EtwManifests next to the ETL file that contains the manifest of all the eventSources in the file.   It also by default reads all manifests from this directory when reading any ETL file next to that ETWManifest Directory.  The result is that you have a persistent copy of the manifests, which will be used if there is no manifest in the file.   This can insure that you have the meta-data you need. 

    • Marked as answer by Dan Moyer Thursday, May 16, 2013 2:39 AM
    Wednesday, May 15, 2013 1:21 AM

All replies

  • I would have expected this behavior if you collect the data with logman but not if you collected the data with PerfView.    It can also happen if the process that contains the EventSource terminates after a long time but before data collection stops.    If indeed you have a problem when the data is collected with PerfView (latest version on the web) and it is not the case above then post the repro instructions and I will fix it. 

    The basic issue is that the metadata associated with EventSource event is logged to the ETL file as just another event.  By default this event gets logged at the start of the trace.  For a circular buffer, these first events eventually get overwritten which is why it fails for Logman.    However PerfView compensates for this by including all the EventSources in the 'rundown' that it does when a trace ends.  This causes the meta-data to be re-emitted and thus present for decoding.  

    Finally PerfView does have some obscure features that can also help.   You can do PerfView userCommand DumpEventSourceManifests FILE.ETL which will create a directory called 'EtwManifests next to the ETL file that contains the manifest of all the eventSources in the file.   It also by default reads all manifests from this directory when reading any ETL file next to that ETWManifest Directory.  The result is that you have a persistent copy of the manifests, which will be used if there is no manifest in the file.   This can insure that you have the meta-data you need. 

    • Marked as answer by Dan Moyer Thursday, May 16, 2013 2:39 AM
    Wednesday, May 15, 2013 1:21 AM
  • First of all THANKS!

    I verified the version of PerfView as suggested and found I was using an earlier version- 1.3.   That version of PerfView was not maintaining the metadata when the buffer wrapped.

    I downloaded the latest version as suggested (1.4.1.0 / build date May 8, 2013).   The latest version works great.

    It's good to be aware of the loss of metadata when using a circular buffer via logman.   I'll keep that in mind when in a scenario where I need to have a fixed size buffer and that PerfView is the appropriate tool to use.

    Also, didn't know about the command DumpEventSourceManifests.   I will be exploring that after completing this post.

    Again, thanks Vance!

    Dan


    Dan Moyer

    Thursday, May 16, 2013 2:39 AM