FAQ: Common Questions for ETW and Windows Event Log RRS feed

  • General discussion

  • Common Questions for ETW and Windows Event Log

    This FAQ applies to operating systems beginning with Windows Vista and Windows Server 2008.


    Q: What are the differences between Windows Event Log, Event Tracing for Windows (ETW), and Windows Software Tracing Pre-Processor (WPP)?

    Event Tracing for Windows (ETW) is a high-speed tracing facility provided by the Windows Operating System. ETW is the core tracing facility in Windows on top of which both the Event Log and WPP are built. ETW supports user-mode applications and kernel-mode device drivers. Additionally, ETW lets you enable or disable tracing dynamically, which makes it possible to perform detailed tracing in production environments without requiring reboots or application restarts. The logging mechanism uses per-processor buffers that are written to disk by a separate writer thread. ETW events include rich metadata, localizable message strings, and schematized (structured) data payloads for easy consumption of event data.

    Windows Event Log is a management-focused event system, designed for system administrators and IT professionals to easily consume events. Tools such as the Event Viewer and Windows PowerShell interact with the Event Log to receive and display events to users. Beginning with Windows Vista, Windows Event Log is built on top of ETW technology. Windows Event Log provides a centralized repository for events which is, in general, always enabled and stored on disk. Windows Event Log lets you subscribe to events using queries to either local or remote machines. Because Windows Event Log is built on top of ETW, the events in Windows Event Log include the same rich metadata, localizable message strings, and schematized (structured) data payloads for easy consumption of event data.

    Beginning with Windows Vista, ETW and Windows Event Log providers use the same APIs to log events. Providers log events to channels, each with a different target audience. The Admin and Operational channel types target administrators and IT professionals, while the Analytic and Debug channels target component developers, performance analysts, and tool builders. Events in the Admin and Operational channels are always enabled and routed to Windows Event Log; Analytic and Debug channels events are not enabled by default.

    Windows Software Trace Pre-Processing (WPP) events are primarily for debugging applications and drivers. WPP does not use the same APIs to log events as ETW and Windows Event Log providers. The events are as easy to add to an application as print-statements, but have the disadvantages of being unstructured, lacking localizable strings, and having a restricted set of data types. They are primarily useful for debugging applications while under development. However, due to the need to use additional decoding data when reading the logged events, they are not easy to consume by end users.


    Q: Is there a FAQ for WPP?

    Please see Software Tracing FAQ on MSDN.


    Q: Where can I find samples of how to add ETW and Windows Event Log events to my code?

    The Windows SDK includes C++ and managed code samples. The samples are located under "%MSSDK%\Samples\WinBase\Eventing”. Each sample folder contains a “readme” file that tells you how to build and run the sample. Use the most recent SDK for the most up-to-date samples.

    The samples contain an XML instrumentation manifest (.man) that describes the events that the sample will log. You can use your favorite text editor to edit the manifest or you can use the EcManGen.exe tool to edit the manifest. The EcManGen tool is available in the SDK located in the "%MSSDK%\bin folder. 

    The message compiler, MC.exe, processes the instrumentation manifest to create instrumentation resource files (.rc) and headers (.h). The compiler is located in the "%MSSDK%\bin folder. The compiler can also generate logging macros (or classes for managed applications) using the events defined in the manifest. The samples use the code generating feature which simplifies writing event providers.

    Look for the following in the header to find what macros (or class methods) to call:

    EventRegister<YourProviderName> : Used to register your provider (at application start).
    EventUnregister<YourProviderName> : Used to unregister your provider (at application finish).
    EventWrite* : One macro (or method) for each event defined in your manifest.

    The compiler can generate code for user-mode and kernel-mode applications, as well as for C# applications. For usage details, see the MC.exe
    documentation on MSDN.


    Q: I’ve created a channel, but my channel is not showing up in the Event Viewer. Why is it missing?

    In the Event Viewer, Analytic and Debug channels aren’t displayed by default. Click on View and make sure ‘Show Analytic and Debug logs’ is enabled. Also, Analytic and Debug channels are not enabled by default. To enable Analytic and Debug channels, right-click on the channel and select ‘Enable Log’. We recommend disabling analytic or debug logs after collecting needed events, since they are often very verbose.

    If the channel is missing or is inaccessible, it may be due to missing channel resources. Resource accessibility can be confirmed by running wevtutil.exe.

    >  wevtutil gp <providername> 

    If your provider is not found, install your manifest using “wevtutil im <manifest>”.

    If an error is returned showing that provider resources are not accessible, check if you compiled your latest instrumentation manifest with MC.exe and that you have included the header file and resource file with your provider binary. Also, make sure that the paths pointing to your provider binary (in the manifest’s <provider> element) are accessible by the Event Log service. If you are using user-defined environment variables to point to your binary, you must restart your computer before the Event Log will recognize the newly defined environment variables.


    Q: I’m trying to log events to the Event Log, but they are not showing up in the Event Viewer. What could I be doing wrong?

    If you are sure that your logging code was executed, please ensure that:

    ·         The <Event> element in your manifest is associated with a channel. Each event must indicate a destination channel if it is to appear in the Event Viewer or Windows PowerShell Get-WinEvent.

    ·         The event data template in your instrumentation manifest matches the event data that your provider is logging for that event.

    ·         The channel is enabled. (Channel enablement can be discovered using the Event Viewer, Windows PowerShell Get-WinEvent, or wevtutil.exe.)

    ·         The binary includes the latest header and resource file. If you change the manifest, you must also remember to compile the instrumentation manifest with MC.exe. When the manifest is changed, you should also uninstall and then reinstall your manifest using Wevtutil. Additionally, make sure to update the new binary which contains the resources in the location specified in the manifest.

    ·         All localized resource files are deployed correctly. If your manifest has any localized messages, please make sure that language specific files, such as MUI files, are also placed correctly.

    If you get an error when you click on an event or channel in the Event Viewer, make sure that the provider’s resource are accessible. To determine whether the resources are accessible, run the “wevtutil gp <providername>” command.


    Q: I’m trying to log an event to an ETW session but they are not showing up in the log file (for example, a TraceRpt dump does not contain my events). What could I be doing wrong?

    The following reasons could explain why the events are not showing up in the log file:

    1.       The ETW session was not started before the event was logged. Use logman.exe to start the ETW session prior to logging the events.

    2.       The event provider was not enabled to the session. Use logman.exe to determine whether the event provider was enabled to the session. If it was not, use logman.exe to enable it to the session.

    3.       You do not have privileges to log events to the session. To determine whether the session is a secured session, run the ”tracelog –q <sessionname>” command. If “Log Mode” includes “Secure”, verify whether you have the right privilege.

    4.       You mixed calls that the message compiler generates with your own calls to EventWrite(). The calls that the message compiler generates use their own callback function to determine whether the session is enabled and whether to log events. If you do not use the registration code created by the message compiler, then the generated code would not know that the session was enabled and would not log the events.


    Q: My events are being logged (for example, TraceRpt is showing events with my Provider GUID and Event IDs), but the event data is not decoded correctly or the events do not include the strings. How do I figure out what’s wrong?

    If the event data is not decoded correctly, it is generally because the event provider was not installed correctly or there is a problem with the decoding resources included in the provider’s binary. Event decoding information is stored in the event provider’s binary as resources and is required to display the events correctly.

    Run the following command to verify that your provider is installed correctly:

    > wevtutil gp <providername>

    If your provider is not found, run the following command to install the provider:

    > wevutil im <manifest>

    If your provider is installed correctly, then check the following to determine whether the binary contains the correct resources for decoding the events:

    1.       Verify that your binary contains decoding information.
    The binary should contain a “1” Message Table resource and a “1” WEVT_TEMPLATE resource.

    2.       Make sure all localized resource files are deployed correctly.
    If your manifest has any localized messages, please make sure that the language specific files, such as MUI files, are deployed correctly.

    3.       If you implemented multiple providers in one binary, please make sure your binary contains decoding information for all of the providers.
    Please verify that all of the providers are declared in one manifest file. If not, please merge all of the providers into one manifest file. Only one XML instrumentation manifest can be used per binary, but the manifest can contain multiple event providers.


    Q: How do I start and stop event tracing sessions for ETW?

    The following show the various ways for starting and stopping an ETW tracing session on Vista and later computers.

    1.       Use the logman.exe tool.

    a.       To use logman to start a session, run the following command:
    > logman start “SessionName” -o "TraceFileName.etl" –p “My-Provider-Name” –ets

    b.      To enable a provider to the session, run the following command:

    > logman update “SessionName” –p “My-Provider-Name” -o "TraceFileName.etl" –ets

    2.       Use the Windows Performance Analysis Tools.

    3.       Use the tracelog.exe tool.

    4.       Use the Reliability and Performance Monitor user interface.

    5.       Use the StartTrace() and ControlTrace() functions or the Performance Logs and Alerts APIs.


    Q: Will event instrumentation impact the performance of my application?

    In general, the performance impact of adding event instrumentation is negligible. If tracing is not enabled, the overhead is a few instructions for each EventWrite function call that are used to bypass the tracing call. If tracing is enabled, EventWrite will incur a few thousand cycles depending on the amount of data being logged and hardware characteristics of the computer. The EventWrite function is a non-blocking call and control will be returned to your application after the OS has received your event data.

    When adding tracing to performance-sensitive code paths, you should measure the overhead with and without tracing to evaluate whether the additional overhead is acceptable. Adding instrumentation will cause the code size to increase, resulting in larger binaries and consequently a bigger memory foot-print, more page-faults and more cache misses. Also, the code path length will increase due to tracing calls being made.


    Q: What is the ETW security model?

    ETW provides security that restricts various operations associated with writing and consuming events.  ETW uses a security descriptor that is associated with a GUID to restrict access to individual sessions and providers. The EventAccessControl function documentation includes a list of the various access rights that ETW supports. Securable operations include session control (starting, stopping, updating, querying), provider registration, and provider enablement.

    For more information, see the EventAccess* functions on MSDN


    Q: What is the Event Log security model?

    Event Log uses the ETW security model to implement channel-level security. The isolation attribute of the Channel element specifies the write access permissions for the channel. The two standard write access permissions are Application and System. To specify your own write access permissions, use Custom isolation.  The following show the security descriptor for Application and System using SDDL.

    Application isolation SDDL:

    SD: O:BAG:BAD:(A;;0x200;;;SY)(A;;0x200;;;BA)(A;;0x200;;;SO)(A;;0x200;;;IU)(A;;0x

    System isolation SDDL:

    SD: O:BAG:BAD:(A;;0x200;;;SY)(A;;0x200;;;BA)(A;;0x200;;;BO)(A;;0x200;;;SU)(A;;0x

    The access attribute of the Channel element specifies the read and clear permissions for the channel. When isolation is “custom”, the access attribute also specifies the write permissions. The default security for access is the same as provided through application isolation.


    Q: What tools are available for consuming events?

    Tools included with the computer:

    • Windows Event Viewer. A user interface that you can use to view events written to Windows Event Log channels. You can create custom queries to query specific events. You can also save logs in XML, CSV, and EVTX format.
    • Wevtutil.exe. A command line tool that you can use to retrieve information about Event Log providers and their events. You can also use Wevtutil to install and uninstall instrumentation manifests, run queries, and to export, archive, and clear logs.
    • Tracerpt.exe. A command line tool that you can use to process event trace logs or real-time data from instrumented ETW and WPP event trace providers. TraceRpt lets you generate trace analysis reports as XML, CSV, or EVTX files.

    Tools included in the Windows SDK:

    • Tracefmt.exe. A command line tool that you can use to format and display trace messages from an event trace log file (.etl) or a real-time trace session. TraceFmt.exe can only consume events from WPP event providers.

    Windows PowerShell V2: contains a new cmdlet (Get-WinEvent) to read events from Windows Event Log.

    Windows Performance Analysis Tools:  Windows Performance Tools (also known as xperf) are designed for analysis of a wide range of performance problems including application start times, boot issues, deferred procedure calls and interrupt activity (DPCs and ISRs), system responsiveness issues, application resource usage, and interrupt storms.


    Q: What are process-private sessions and when should I use them?

    Processes that do not require data from the kernel can eliminate the overhead associated with kernel-mode transitions by using a private event tracing session. A private event tracing session is a user-mode event tracing session that runs in the same process as its event provider. The memory for buffers comes from the process memory.

    For more information about process-private sessions, see the MSDN documentation on Logging Mode Constants which are passed to the StartTrace API.

    Q: Where do I go to find out more information about Windows Event Forwarding and the Windows Event Collector?

    Windows Event Collector service allows administrators to receive and store events that are forwarded from a remote computer. You can create subscriptions for events using WS-Mananagement protocol. You can find more information on MSDN.

    Tuesday, May 5, 2009 9:50 PM