locked
Memory crash with FwpsInjectTransportReceiveAsync0 RRS feed

  • Question

  • My driver, derived and slightly modified from the inspect sample, crashes randomly on a Windows 7 SP1 64-bit computer. It can run several hours without a hitch.
    I don't understand why sometimes FwpsInjectTransportReceiveAsync0 crashes. Any thoughts ?




    Microsoft (R) Windows Debugger Version 6.12.0002.633 AMD64
    Copyright (c) Microsoft Corporation. All rights reserved.


    Loading Dump File [C:\Windows\Minidump\042312-41090-01.dmp]
    Mini Kernel Dump File: Only registers and stack trace are available

    WARNING: Non-directory path: 'D:\projects\TEST\inspect\objchk_win7_amd64\amd64\Inspect.sys'
    Symbol search path is: http://msdl.microsoft.com/download/symbols;D:\projects\TEST\inspect\objchk_win7_amd64\amd64
    Executable search path is: D:\projects\TEST\inspect\objchk_win7_amd64\amd64\Inspect.sys
    Windows 7 Kernel Version 7601 (Service Pack 1) MP (4 procs) Free x64
    Product: WinNt, suite: TerminalServer SingleUserTS
    Built by: 7601.17790.amd64fre.win7sp1_gdr.120305-1505
    Machine Name:
    Kernel base = 0xfffff800`02e50000 PsLoadedModuleList = 0xfffff800`03094650
    Debug session time: Mon Apr 23 22:39:12.478 2012 (UTC + 2:00)
    System Uptime: 0 days 2:52:53.617
    Loading Kernel Symbols
    ...............................................................
    ................................................................
    ..............................................
    Loading User Symbols
    Loading unloaded module list
    .........
    *******************************************************************************
    *                                                                             *
    *                        Bugcheck Analysis                                    *
    *                                                                             *
    *******************************************************************************

    Use !analyze -v to get detailed debugging information.

    BugCheck D1, {0, 2, 0, fffff88001750452}

    Probably caused by : NETIO.SYS ( NETIO!NetioDereferenceNetBufferList+86 )

    Followup: MachineOwner
    ---------

    3: kd> !analyze -v
    *******************************************************************************
    *                                                                             *
    *                        Bugcheck Analysis                                    *
    *                                                                             *
    *******************************************************************************

    DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
    An attempt was made to access a pageable (or completely invalid) address at an
    interrupt request level (IRQL) that is too high.  This is usually
    caused by drivers using improper addresses.
    If kernel debugger is available get stack backtrace.
    Arguments:
    Arg1: 0000000000000000, memory referenced
    Arg2: 0000000000000002, IRQL
    Arg3: 0000000000000000, value 0 = read operation, 1 = write operation
    Arg4: fffff88001750452, address which referenced memory

    Debugging Details:
    ------------------


    READ_ADDRESS: GetPointerFromAddress: unable to read from fffff800030fe100
     0000000000000000

    CURRENT_IRQL:  2

    FAULTING_IP:
    tcpip! ?? ::FNODOBFM::`string'+5d64
    fffff880`01750452 488b01          mov     rax,qword ptr [rcx]

    CUSTOMER_CRASH_COUNT:  1

    DEFAULT_BUCKET_ID:  VISTA_DRIVER_FAULT

    BUGCHECK_STR:  0xD1

    PROCESS_NAME:  System

    TRAP_FRAME:  fffff8800681ad10 -- (.trap 0xfffff8800681ad10)
    NOTE: The trap frame does not contain all registers.
    Some register values may be zeroed or incorrect.
    rax=fffffa80079cdc20 rbx=0000000000000000 rcx=0000000000000000
    rdx=fffff88002fd5180 rsi=0000000000000000 rdi=0000000000000000
    rip=fffff88001750452 rsp=fffff8800681aea0 rbp=0000000000000000
     r8=0000000000006888  r9=00000000000000d0 r10=fffff80002e50000
    r11=000000000000029d r12=0000000000000000 r13=0000000000000000
    r14=0000000000000000 r15=0000000000000000
    iopl=0         nv up ei ng nz na pe nc
    tcpip! ?? ::FNODOBFM::`string'+0x5d64:
    fffff880`01750452 488b01          mov     rax,qword ptr [rcx] ds:a670:00000000`00000000=????????????????
    Resetting default scope

    LAST_CONTROL_TRANSFER:  from fffff80002ecc229 to fffff80002eccc80

    STACK_TEXT:  
    fffff880`0681abc8 fffff800`02ecc229 : 00000000`0000000a 00000000`00000000 00000000`00000002 00000000`00000000 : nt!KeBugCheckEx
    fffff880`0681abd0 fffff800`02ecaea0 : fffff880`0681ae20 fffffa80`082b9320 00000000`00000000 fffffa80`08142a70 : nt!KiBugCheckDispatch+0x69
    fffff880`0681ad10 fffff880`01750452 : fffffa80`08142a70 00000000`00000000 fffffa80`206c644d fffffa80`06d5a670 : nt!KiPageFault+0x260
    fffff880`0681aea0 fffff880`01403336 : fffffa80`08142a70 00000000`01719d3a 00000000`00000000 00000000`00000000 : tcpip! ?? ::FNODOBFM::`string'+0x5d64
    fffff880`0681aef0 fffff880`01402a92 : 00000000`00000000 fffffa80`0769db10 00000000`00000000 00000000`00000011 : NETIO!NetioDereferenceNetBufferList+0x86
    fffff880`0681af20 fffff880`017194c2 : 00000000`00000000 fffffa80`0769dc00 fffff880`0681af00 fffff880`0681b040 : NETIO!NetioDereferenceNetBufferListChain+0x332
    fffff880`0681aff0 fffff880`017170af : 00000000`faffffef fffffa80`0783d000 fffff880`018399a0 00000000`069a8a01 : tcpip!IppReceiveHeaderBatch+0x3c3
    fffff880`0681b0d0 fffff880`018020a2 : fffffa80`081b3270 00000000`00000000 fffffa80`069a8a01 00000000`00000001 : tcpip!IpFlcReceivePackets+0x64f
    fffff880`0681b2d0 fffff880`018d5af6 : fffffa80`08423002 fffffa80`08423010 00000000`00000002 00000000`00000000 : tcpip!IppInspectInjectReceive+0xf2
    fffff880`0681b310 fffff880`07bb41bc : fffffa80`07c9cc30 fffffa80`06cbc060 00000000`00000000 fffff880`00000000 : fwpkclnt!FwpsInjectTransportReceiveAsync0+0x256
    fffff880`0681b3c0 fffff880`07bb5134 : fffffa80`069a8ab0 fffffa80`068a8040 00000000`00000001 fffffa80`068a8040 : Inspect!TLInspectCloneReinjectInbound+0x30c [d:\projects\test\inspect\inspect.c @ 1424]
    fffff880`0681b470 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : Inspect!TLInspectWorker+0x8b4 [d:\projects\test\inspect\inspect.c @ 2152]


    STACK_COMMAND:  kb

    FOLLOWUP_IP:
    NETIO!NetioDereferenceNetBufferList+86
    fffff880`01403336 4885ff          test    rdi,rdi

    SYMBOL_STACK_INDEX:  4

    SYMBOL_NAME:  NETIO!NetioDereferenceNetBufferList+86

    FOLLOWUP_NAME:  MachineOwner

    MODULE_NAME: NETIO

    IMAGE_NAME:  NETIO.SYS

    DEBUG_FLR_IMAGE_TIMESTAMP:  4ce79381

    FAILURE_BUCKET_ID:  X64_0xD1_NETIO!NetioDereferenceNetBufferList+86

    BUCKET_ID:  X64_0xD1_NETIO!NetioDereferenceNetBufferList+86

    Followup: MachineOwner
    ---------

    Monday, April 23, 2012 9:31 PM

All replies

  • In some cases the crash becomes "Attempt to free pool which was already freed" in ExFreePoolWithTag called from FreePendedPacket called from TLInspectInjectComplete called from FwpsInjectTransportReceiveAsync0.

    The number of CPU core in the VM impacts the behavior. More there are cores, more the crashes are easy to reproduce.


    • Edited by OlivierMSDN Wednesday, April 25, 2012 8:57 PM
    Wednesday, April 25, 2012 8:56 PM
  • Sustained engineering is aware of the issue and is investigating.  I will post back when any news emerges.

    [REF WinSE 398584]

    Thanks,


    Dusty Harper [MSFT]
    Microsoft Corporation
    ------------------------------------------------------------
    This posting is provided "AS IS", with NO warranties and confers NO rights
    ------------------------------------------------------------


    Friday, April 27, 2012 5:01 PM
    Moderator
  • Thanks for the feedback. As there are no crash under "verifier", it's even harder to progress.
    Same issue with Windows 7 and 2008 R2.
    Friday, April 27, 2012 6:01 PM