none
DPC_WATCHDOG_VIOLATION (0x133) bug check RRS feed

  • Question

  • Hello,

    We are experiencing many 0x133 BSODs on our product with many different stacks.

    All of them are due to a  single DPC or ISR that exceeded its time allotment (>280 ticks)

    There are many different stacks, but The common to all of them is that the BSOD occur when our driver release a spin lock.

    This trigger the verifier to check the spin lock, which then trigger the 0x133 BSOD

    Here is the OS stack part which cause the 0x133 BSOD: (OS version 9200.16384.amd64fre.win8_rtm.120725-1247)

    nt!KeBugCheckEx
    nt! ?? ::FNODOBFM::`string'
    nt!KeUpdateRunTime
    hal!HalpTimerClockInterrupt
    nt!KiInterruptDispatchLBControl
    nt!RtlpWalkFrameChain
    nt!RtlWalkFrameChain
    nt!RtlCaptureStackBackTrace
    nt!VerifierKeReleaseSpinLock
    NETwew00!releaseSpinLock
    ...

    Questions:

    1. Can you explain if the cause to the 0x133 BSOD is due to a long DPC runtime of our driver (NETwew00) or due to a long DPC runtime of the verifier which seems to update some verifier databases upon getting spinlock release from the driver?

    2. What windbg commands may help us continue to debug this issue?

    Thanks

    Thursday, November 29, 2012 6:36 PM

All replies

  • Without a full !analyze -v almost no crash can be debugged.  You give us no information that is useful in answering your questions.


    Don Burn Windows Filesystem and Driver Consulting Website: http://www.windrvr.com Blog: http://msmvps.com/blogs/WinDrvr

    Thursday, November 29, 2012 6:41 PM
  • I will gladly add the Analyze -v info..

    The 0x133 is right after the driver calls NdisReleaseSpinLock

    I suspect that the verifier is causing the DPC watchdog issue and not the driver calling spinlock free.

    Thanks

    *******************************************************************************
    *                                                                             *
    *                        Bugcheck Analysis                                    *
    *                                                                             *
    *******************************************************************************
    DPC_WATCHDOG_VIOLATION (133)
    The DPC watchdog detected a prolonged run time at an IRQL of DISPATCH_LEVEL
    or above.
    Arguments:
    Arg1: 0000000000000000, A single DPC or ISR exceeded its time allotment. The offending
     component can usually be identified with a stack trace.
    Arg2: 0000000000000281, The DPC time count (in ticks).
    Arg3: 0000000000000280, The DPC time allotment (in ticks).
    Arg4: 0000000000000000
    Debugging Details:
    ------------------
    DPC_TIMEOUT_TYPE:  SINGLE_DPC_TIMEOUT_EXCEEDED
    DEFAULT_BUCKET_ID:  WIN8_DRIVER_FAULT
    BUGCHECK_STR:  0x133
    PROCESS_NAME:  NICMasterApp.e
    CURRENT_IRQL:  d
    LAST_CONTROL_TRANSFER:  from fffff800674465df to fffff800672ed040
    STACK_TEXT:  
    fffff880`02b6fd88 fffff800`674465df : 00000000`00000133 00000000`00000000 00000000`00000281 00000000`00000280 : nt!KeBugCheckEx
    fffff880`02b6fd90 fffff800`67317f11 : fffff880`00002148 fffff880`02b40180 fffff880`02b6fef0 fffff780`00000320 : nt! ?? ::FNODOBFM::`string'+0x13ba4
    fffff880`02b6fe10 fffff800`67228e84 : ffffffff`ffd0e9a0 fffffa80`04200502 00000005`00000000 00000011`00000001 : nt!KeUpdateRunTime+0x51
    fffff880`02b6fe40 fffff800`672e64ae : fffff880`02b71000 fffff880`02b70780 fffffa80`04200580 fffff880`0420e000 : hal!HalpTimerClockInterrupt+0x50
    fffff880`02b6fe70 fffff800`672823dd : fffff880`00005583 fffff880`02b70150 fffff880`02b70130 fffff880`02b70740 : nt!KiInterruptDispatchLBControl+0x1ce
    fffff880`02b70000 fffff800`6727f72a : fffff880`02b6b000 fffff880`02b71000 fffff880`00000000 fffff880`042e4bc0 : nt!RtlpWalkFrameChain+0x491
    fffff880`02b706e0 fffff800`6727f7e0 : 00000000`00000002 fffffa80`03657208 00000000`00000000 00000000`00000000 : nt!RtlWalkFrameChain+0x5e
    fffff880`02b70710 fffff800`678c1f36 : fffff880`04e3b400 fffff880`02b70c30 00000000`00000002 4430725f`00000000 : nt!RtlCaptureStackBackTrace+0x44
    fffff880`02b70740 fffff880`0425eff6 : fffff980`05c88b90 00000000`00000000 00000000`000001b0 00000000`4430725f : nt!VerifierKeReleaseSpinLock+0xd2
    fffff880`02b70780 fffff880`04ae8c19 : fffff880`02b70888 00000000`000001b0 fffff880`4430725f fffff880`04d2edf0 : NETwew00!memDbgAlloc+0xe6 [d:\buildviews\wfdrv_private\wfdrv_r12404_04669\mwg_driver\shiloh\win_driver\common\utilities\mem.c @ 330]
    fffff880`02b707e0 fffff880`04ae9c68 : fffff880`02b70888 5d33307c`4430725f fffff880`04d2ef40 fffff980`00000054 : NETwew00!prvRxDataDbgAllocateMdu+0x1c9 [d:\buildviews\wfdrv_private\wfdrv_r12404_04669\mwg_driver\shiloh\win_driver\rxdata\inc\rxdatautil.h @ 289]
    fffff880`02b70850 fffff880`04ae2826 : fffff880`04e44160 fffff880`02b709d0 00000000`00000000 00000000`00000000 : NETwew00!rxDataProcessMPDUFrame+0x408 [d:\buildviews\wfdrv_private\wfdrv_r12404_04669\mwg_driver\shiloh\win_driver\rxdata\rxdataprocessmpduframe.c @ 85]
    fffff880`02b708e0 fffff880`042e3767 : fffff880`02b709d0 fffff880`00000003 fffff880`04be1280 fffff880`04be1200 : NETwew00!rxDataCbReceiveComplete+0xc6 [d:\buildviews\wfdrv_private\wfdrv_r12404_04669\mwg_driver\shiloh\win_driver\rxdata\rxdata.c @ 447]
    fffff880`02b70930 fffff880`042e4bc0 : fffff980`05076e34 fffff980`05c5aab4 fffff880`00000053 fffff880`04be10ff : NETwew00!prvRfdQueueDispatch+0x647 [d:\buildviews\wfdrv_private\wfdrv_r12404_04669\mwg_driver\shiloh\win_driver\alon\rfdqueue.c @ 751]
    fffff880`02b70a00 fffff880`042dd4be : fffff980`05076e34 fffff980`05c5aab4 fffff880`400100ff 00000000`00000000 : NETwew00!rfdQueueProcessFragments+0x420 [d:\buildviews\wfdrv_private\wfdrv_r12404_04669\mwg_driver\shiloh\win_driver\alon\rfdqueue.c @ 534]
    fffff880`02b70ad0 fffff880`042c8184 : fffff980`059c6e44 fffff980`05c5aab4 00000000`00000023 00000000`00000004 : NETwew00!isrHandlerRoutineInta+0x5fe [d:\buildviews\wfdrv_private\wfdrv_r12404_04669\mwg_driver\shiloh\win_driver\alon\isr.c @ 889]
    fffff880`02b70b60 fffff880`04b50ba9 : fffff980`05c5aab4 00000000`0000832b fffffa80`05bdb4f8 fffff880`01afcc0a : NETwew00!alonExInterruptHandlerRoutine+0x34 [d:\buildviews\wfdrv_private\wfdrv_r12404_04669\mwg_driver\shiloh\win_driver\alon\alonex.c @ 68]
    fffff880`02b70b90 fffff880`018dae2e : fffff980`0ac78d14 fffff800`00000000 00000000`00000000 fffff880`02b70c78 : NETwew00!oscHandleMsiInterrupt+0x39 [d:\buildviews\wfdrv_private\wfdrv_r12404_04669\mwg_driver\shiloh\win_driver\oscontroller\osccore\osccommon.c @ 499]
    fffff880`02b70bd0 fffff880`018daf3a : fffff980`05c88e78 00000000`ffffffff fffff980`05c88b90 000000c9`ba4edef1 : ndis!ndisMiniportDpc+0xfe
    fffff880`02b70c70 fffff800`672e3498 : fffff880`02b42f00 fffff800`00001000 fffffa80`05bdb598 fffff880`02b70f44 : ndis!ndisInterruptDpc+0x9e
    fffff880`02b70d00 fffff800`67313d50 : fffff880`02b40180 fffff880`038a59c0 fffffa80`0594f640 00000000`00000004 : nt!KiExecuteAllDpcs+0x198
    fffff880`02b70e40 fffff800`67313745 : 00000000`00000000 fffff880`02b40180 fffff880`06c29cc0 fffff880`03156540 : nt!KiRetireDpcList+0xd0
    fffff880`02b70fb0 fffff800`67313549 : fffff880`06c29bd0 00000000`00000010 ffff2458`6ee999e6 00000000`0077e950 : nt!KxRetireDpcList+0x5
    fffff880`06c29c00 fffff800`673ae4d5 : 00000000`0299afe8 fffff800`672e61a3 fffff880`06c29cc0 00000000`0077e950 : nt!KiDispatchInterruptContinue
    fffff880`06c29c30 fffff800`672e61a3 : fffff880`06c29cc0 00000000`0077e950 fffff880`06c29cc0 fffff880`00000000 : nt!KiDpcInterruptBypass+0x25
    fffff880`06c29c40 00000000`6bde64c2 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiInterruptDispatch+0x273
    00000000`0087c898 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x6bde64c2
    STACK_COMMAND:  kb
    FOLLOWUP_IP: 
    NETwew00!memDbgAlloc+e6 [d:\buildviews\wfdrv_private\wfdrv_r12404_04669\mwg_driver\shiloh\win_driver\common\utilities\mem.c @ 330]
    fffff880`0425eff6 488b442460      mov     rax,qword ptr [rsp+60h]
    FAULTING_SOURCE_LINE:  d:\buildviews\wfdrv_private\wfdrv_r12404_04669\mwg_driver\shiloh\win_driver\common\utilities\mem.c
    FAULTING_SOURCE_FILE:  d:\buildviews\wfdrv_private\wfdrv_r12404_04669\mwg_driver\shiloh\win_driver\common\utilities\mem.c
    FAULTING_SOURCE_LINE_NUMBER:  330
    FAULTING_SOURCE_CODE:  
       328:             NdisReleaseSpinLock(&memSpinLock);
       329: 
    >  330:             if (NULL == *pPtr)
       331:             {
    SYMBOL_STACK_INDEX:  9
    SYMBOL_NAME:  NETwew00!memDbgAlloc+e6
    FOLLOWUP_NAME:  MachineOwner
    MODULE_NAME: NETwew00
    IMAGE_NAME:  NETwew00.sys
    DEBUG_FLR_IMAGE_TIMESTAMP:  50b51e88
    BUCKET_ID_FUNC_OFFSET:  e6
    FAILURE_BUCKET_ID:  0x133_VRF_DPC_NETwew00!memDbgAlloc
    BUCKET_ID:  0x133_VRF_DPC_NETwew00!memDbgAlloc
    Followup: MachineOwner
    ---------


    • Edited by Amitbern2 Friday, November 30, 2012 9:29 PM
    Friday, November 30, 2012 7:28 PM
  • Perhaps the system does not have adequate cooling and the clock is speed is severely throttled down. This can be a possible cause for the bugcheck.

    Run LatencyMon and see how your driver compares to other drivers in the system

    //Daniel

    Saturday, December 1, 2012 4:09 AM
  • The clock speed is throttled down only on my driver?, this doesn't make any sense..

    Only when the driver calls to release spinlock?

    can you elaborate on that?


    • Edited by Amitbern2 Saturday, December 1, 2012 11:26 AM
    Saturday, December 1, 2012 11:24 AM
  • DPC's are called on the transition from DISPATCH_LEVEL to a lower IRQL.  Releasing a spinlock is one way this happens.  While your driver is definitly suspicious here, there is nothing that says whose DPC is causing the problem.  

    Do you have a DPC that does a lot of work?  If so how hard would it be to split into two parts and have the first DPC invoke the second?  This would tell you if there is a problem.


    Don Burn Windows Filesystem and Driver Consulting Website: http://www.windrvr.com Blog: http://msmvps.com/blogs/WinDrvr

    Saturday, December 1, 2012 12:28 PM
  • >The clock speed is throttled down only on my driver ?

    No, perhaps your driver is the one with the most lengthy DPC routine so this is the one triggering the bugcheck.  This is why I said, just run LatencyMon and you get a good picture of what is running in your system and which DPCs are executing for too long.

    >Only when the driver calls to release spinlock?

    >There are many different stacks ...

    You seem to have answered that one yourself.

    //Daniel

     


    Saturday, December 1, 2012 12:58 PM
  • With my HP Pavillion 6985 Laptop, the above problem was with the MICROSOFT KERNEL DEBUG NETWORK ADAPTER under DEVICE MANAGER>NETWORK ADAPTERS. Delete or disable.
    Tuesday, July 30, 2013 5:42 PM
  • Please explain how that was the problem.  Such a report is highly doubful.


    Don Burn Windows Filesystem and Driver Consulting Website: http://www.windrvr.com Blog: http://msmvps.com/blogs/WinDrvr

    Tuesday, July 30, 2013 5:46 PM