locked
i think hotfix needed for FwpsStreamInjectAsync RRS feed

  • Question

  • So, I am at the stream layer port 80; clone, blocking and reinjecting...everything works fine until I hit a websites that returns HTTP 204, which means no content body...it doesn't /always/ crash on these, but, when I blue screen, the last website it tried to reinject has this response...

    http://rmx-match.dotomi.com/dmm/rmx/match?xid=.AFf.EchsCgaMLVruFZRTKrJ


    *** Fatal System Error: 0x000000d1
                           (0x000000000000003C,0x0000000000000002,0x0000000000000001,0xFFFFF800ECA53192)

    Break instruction exception - code 80000003 (first chance)

    A fatal system error has occurred.
    Debugger entered on first try; Bugcheck callbacks have not been invoked.

    A fatal system error has occurred.

    Connected to Windows 8.1 9600 x64 target at (Tue Aug 23 21:37:31.818 2016 (UTC - 7:00)), ptr64 TRUE
    Loading Kernel Symbols
    .......................

    Press ctrl-c (cdb, kd, ntsd) or ctrl-break (windbg) to abort symbol loads that take too long.
    Run !sym noisy before .reload to track down problems loading symbols.

    ........................................
    ................................................................
    ...............................
    Loading User Symbols

    Loading unloaded module list
    ............
    *******************************************************************************
    *                                                                             *
    *                        Bugcheck Analysis                                    *
    *                                                                             *
    *******************************************************************************

    Use !analyze -v to get detailed debugging information.

    BugCheck D1, {3c, 2, 1, fffff800eca53192}

    Probably caused by : e1i63x64.sys ( e1i63x64!RECEIVE::RxIndicateNBLs+d4 )

    Followup:     MachineOwner
    ---------

    nt!DbgBreakPointWithStatus:
    fffff800`749dfe90 cc              int     3
    kd> !analyze -v
    *******************************************************************************
    *                                                                             *
    *                        Bugcheck Analysis                                    *
    *                                                                             *
    *******************************************************************************

    DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
    An attempt was made to access a pageable (or completely invalid) address at an
    interrupt request level (IRQL) that is too high.  This is usually
    caused by drivers using improper addresses.
    If kernel debugger is available get stack backtrace.
    Arguments:
    Arg1: 000000000000003c, memory referenced
    Arg2: 0000000000000002, IRQL
    Arg3: 0000000000000001, value 0 = read operation, 1 = write operation
    Arg4: fffff800eca53192, address which referenced memory

    Debugging Details:
    ------------------


    BUGCHECK_P1: 3c

    BUGCHECK_P2: 2

    BUGCHECK_P3: 1

    BUGCHECK_P4: fffff800eca53192

    WRITE_ADDRESS:  000000000000003c 

    CURRENT_IRQL:  2

    FAULTING_IP: 
    tcpip!TcpBeginTcbSend+732
    fffff800`eca53192 f0ff403c        lock inc dword ptr [rax+3Ch]

    CPU_COUNT: 1

    CPU_MHZ: 6a0

    CPU_VENDOR:  GenuineIntel

    CPU_FAMILY: 6

    CPU_MODEL: 45

    CPU_STEPPING: 1

    DEFAULT_BUCKET_ID:  WIN8_DRIVER_FAULT

    BUGCHECK_STR:  AV

    PROCESS_NAME:  System

    ANALYSIS_VERSION: 10.0.10240.9 amd64fre

    DPC_STACK_BASE:  FFFFF80076370FB0

    TRAP_FRAME:  fffff8007636f800 -- (.trap 0xfffff8007636f800)
    NOTE: The trap frame does not contain all registers.
    Some register values may be zeroed or incorrect.
    rax=0000000000000000 rbx=0000000000000000 rcx=ffffcf81cfd70ea0
    rdx=0000000000000000 rsi=0000000000000000 rdi=0000000000000000
    rip=fffff800eca53192 rsp=fffff8007636f990 rbp=fffff8007636fa90
     r8=ffffcf81cfd70e90  r9=ffffcf81cfd70dd0 r10=ffffcf81cfd70dd0
    r11=ffffcf81cfd70fe8 r12=0000000000000000 r13=0000000000000000
    r14=0000000000000000 r15=0000000000000000
    iopl=0         nv up ei pl nz na pe nc
    tcpip!TcpBeginTcbSend+0x732:
    fffff800`eca53192 f0ff403c        lock inc dword ptr [rax+3Ch] ds:00000000`0000003c=????????
    Resetting default scope

    LAST_CONTROL_TRANSFER:  from fffff80074a6898e to fffff800749dfe90

    STACK_TEXT:  
    fffff800`7636ef08 fffff800`74a6898e : 00000000`00000000 00000000`00000000 fffff800`7636f070 fffff800`7495f7a4 : nt!DbgBreakPointWithStatus
    fffff800`7636ef10 fffff800`74a6829f : 00000000`00000003 fffff800`7636f070 fffff800`749e7290 00000000`000000d1 : nt!KiBugCheckDebugBreak+0x12
    fffff800`7636ef70 fffff800`749d93a4 : fffff800`7636fa48 00000000`0004244e 00000000`00000000 fffff800`749233e7 : nt!KeBugCheck2+0x8ab
    fffff800`7636f680 fffff800`749e4de9 : 00000000`0000000a 00000000`0000003c 00000000`00000002 00000000`00000001 : nt!KeBugCheckEx+0x104
    fffff800`7636f6c0 fffff800`749e363a : 00000000`00000001 fffff800`7636fa48 fffff800`7636b000 fffff800`76371000 : nt!KiBugCheckDispatch+0x69
    fffff800`7636f800 fffff800`eca53192 : 00000000`fffffffe ffffe000`391c7a70 ffff7a41`47f1303e ffffe000`377e5490 : nt!KiPageFault+0x23a
    fffff800`7636f990 fffff800`eca51e52 : 00000000`00020000 fffff800`ebf27ce7 ffffe000`37d19008 00000000`00000001 : tcpip!TcpBeginTcbSend+0x732
    fffff800`7636fc70 fffff800`eca744ab : 00000000`00000000 00000000`00000001 ffffe000`38183010 ffffcf81`ce38adf0 : tcpip!TcpTcbSend+0x226
    fffff800`7636ffc0 fffff800`eca4991c : ffffd001`0a217c2c 00000000`00046dbc 00000000`00000000 00000000`00000000 : tcpip!TcpFlushDelay+0x20a
    fffff800`76370070 fffff800`eca43423 : ffffe000`37d1dbb0 00000000`00005000 00000000`0000b5c5 fffff800`74b4b5c5 : tcpip!TcpPreValidatedReceive+0x3cc
    fffff800`76370170 fffff800`eca76e32 : ffffe000`37fc63d0 fffff800`76370600 00000000`00000006 ffffcf81`cfcb0006 : tcpip!IpFlcReceivePreValidatedPackets+0x649
    fffff800`76370350 fffff800`74932fc3 : 00000000`00000001 00000000`00000000 ffffe000`37d2eb10 fffff800`7636b000 : tcpip!FlReceiveNetBufferListChainCalloutRoutine+0x102
    fffff800`76370480 fffff800`eca77076 : fffff800`eca76d30 fffff800`763705a0 00000000`00000010 00000000`00000801 : nt!KeExpandKernelStackAndCalloutInternal+0xf3
    fffff800`76370570 fffff800`ec887a53 : 00000000`00000000 fffff800`76370651 00000000`00000001 fffff800`eca54550 : tcpip!FlReceiveNetBufferListChain+0xb6
    fffff800`763705f0 fffff800`ec887e7f : 00000000`00000001 00000000`00000000 00000000`00000000 00000000`00000001 : ndis!ndisMIndicateNetBufferListsToOpen+0x123
    fffff800`763706b0 fffff800`ec8886b2 : ffffe000`380e91a0 00000000`00000001 fffff800`ec894540 00000000`00000000 : ndis!ndisMTopReceiveNetBufferLists+0x22f
    fffff800`76370740 fffff800`ed7861c4 : ffffe000`38115000 fffff800`ed786efc ffffe000`38115e00 ffffcf81`ce322df0 : ndis!NdisMIndicateReceiveNetBufferLists+0x732
    fffff800`76370930 fffff800`ed786a9d : 00000000`00000001 ffffcf81`ce322df0 ffffe000`38115000 00000000`00000001 : e1i63x64!RECEIVE::RxIndicateNBLs+0xd4
    fffff800`76370970 fffff800`ed779150 : 00000000`00000000 ffffcf81`cda4eff0 00000000`00000000 ffff0001`00000000 : e1i63x64!RECEIVE::RxProcessInterrupts+0x19d
    fffff800`763709f0 fffff800`ed77957e : ffffcf81`cda4eff0 ffffe000`38115000 ffff0001`00000000 ffff0001`00000000 : e1i63x64!INTERRUPT::MsgIntDpcTxRxProcessing+0x1c0
    fffff800`76370a60 fffff800`ed778b78 : fffff800`76370b79 ffff0001`00000000 00000000`00000000 ffffe000`380e91a0 : e1i63x64!INTERRUPT::MsgIntMessageInterruptDPC+0x13e
    fffff800`76370ac0 fffff800`ec889e12 : 00000000`00000000 fffff800`ed0b3d08 ffffe000`380c3502 fffff800`748c3e17 : e1i63x64!INTERRUPT::MiniportMessageInterruptDPC+0x28
    fffff800`76370b00 fffff800`748c3910 : 00000000`00000000 fffff800`7488b000 fffff800`74b3f480 ffffe000`35da0f44 : ndis!ndisInterruptDpc+0x1a3
    fffff800`76370be0 fffff800`748c2c57 : 00000000`00000000 ffffe000`39ee7080 fffff800`74b88180 00000000`00000000 : nt!KiExecuteAllDpcs+0x1b0
    fffff800`76370d30 fffff800`749dc3d5 : 00000000`00000000 fffff800`74b88180 00000000`00000000 ffffe000`391c7a70 : nt!KiRetireDpcList+0xd7
    fffff800`76370fb0 fffff800`749dc1d9 : 0003eeec`00780000 00000000`00000010 ffffd001`0af334a0 ffffcf81`00000080 : nt!KxRetireDpcList+0x5
    ffffd001`0af333f0 fffff800`749de2fa : 00000000`00000001 ffffe000`38cbf140 ffffe000`38204410 00000000`00000000 : nt!KiDispatchInterruptContinue
    ffffd001`0af33420 fffff800`748be652 : ffffe000`391c7ae0 ffffe000`391c7a70 ffffe000`39f64740 ffffd001`0af336e0 : nt!KiDpcInterrupt+0xca
    ffffd001`0af335b0 fffff800`eca5222c : ffffe000`37ce9c00 ffffd001`0af33601 00000000`00000000 00000000`0000026b : nt!KeReleaseSpinLock+0x22
    ffffd001`0af335e0 fffff800`eca47bc5 : ffffe000`391c7a70 00000000`00000000 ffffd001`0af339c2 ffffe000`391c7a70 : tcpip!TcpTcbSend+0x600
    ffffd001`0af33930 fffff800`eca4780c : 00000000`00046dbc 00000000`00001000 00000000`00000001 ffffd001`0af33ad0 : tcpip!TcpEnqueueTcbSendOlmNotifySendComplete+0xa5
    ffffd001`0af33960 fffff800`ec55e6a6 : 00000000`00000000 00000000`00000000 ffffcf81`cfd2adf0 00000000`00000000 : tcpip!TcpEnqueueTcbSend+0x2ac
    ffffd001`0af33a60 fffff800`ec55ec77 : ffffe000`36ec1de0 ffffe000`391c7a70 00000000`00000000 fffff800`748b4950 : NETIO!StreamInjectRequestsToStack+0x24a
    ffffd001`0af33b40 fffff800`ec55ed82 : ffffe000`36ec1de0 ffffe000`3a00b980 ffffe000`00000001 ffff79a3`f5105fba : NETIO!StreamPermitDataHelper+0x5f
    ffffd001`0af33b70 fffff800`7493f304 : ffffcf81`cfa80fb0 ffffcf81`cd9d6f00 00000000`00000000 fffff800`ec55ecc0 : NETIO!StreamPermitRemoveDataWorkerRoutine+0xc2
    ffffd001`0af33be0 fffff800`7493fa2f : fffff800`748b46d0 fffff800`7493f284 ffffe000`39751040 00000000`00000000 : nt!IopProcessWorkItem+0x80
    ffffd001`0af33c50 fffff800`74985c10 : ffffe000`35d33880 ffffe000`39751040 00000000`00000080 ffffe000`39751040 : nt!ExpWorkerThread+0x69f
    ffffd001`0af33d00 fffff800`749df8c6 : fffff800`74b88180 ffffe000`39751040 ffffe000`35d33880 d82b48f0`508b0000 : nt!PspSystemThreadStartup+0x58
    ffffd001`0af33d60 00000000`00000000 : ffffd001`0af34000 ffffd001`0af2e000 00000000`00000000 00000000`00000000 : nt!KiStartSystemThread+0x16


    STACK_COMMAND:  kb

    FOLLOWUP_IP: 
    e1i63x64!RECEIVE::RxIndicateNBLs+d4
    fffff800`ed7861c4 40f6c702        test    dil,2

    SYMBOL_STACK_INDEX:  11

    SYMBOL_NAME:  e1i63x64!RECEIVE::RxIndicateNBLs+d4

    FOLLOWUP_NAME:  MachineOwner

    MODULE_NAME: e1i63x64

    IMAGE_NAME:  e1i63x64.sys

    DEBUG_FLR_IMAGE_TIMESTAMP:  51496739

    BUCKET_ID_FUNC_OFFSET:  d4

    FAILURE_BUCKET_ID:  AV_VRF_e1i63x64!RECEIVE::RxIndicateNBLs

    BUCKET_ID:  AV_VRF_e1i63x64!RECEIVE::RxIndicateNBLs

    PRIMARY_PROBLEM_CLASS:  AV_VRF_e1i63x64!RECEIVE::RxIndicateNBLs

    ANALYSIS_SOURCE:  KM

    FAILURE_ID_HASH_STRING:  km:av_vrf_e1i63x64!receive::rxindicatenbls

    FAILURE_ID_HASH:  {da165245-bf6e-90ea-00d7-b47bffc449f5}

    Followup:     MachineOwner
    ---------
    • Edited by DeyBack Wednesday, August 24, 2016 5:34 AM
    Wednesday, August 24, 2016 5:33 AM

Answers

  • oh...I believe I fixed the problem..

    so I was returning FWPS_STREAM_ACTION_NEED_MORE_DATA ---and not checking if the data length changed or remained constant..so I guess it went into an infinite loop and crashed the stream

    • Marked as answer by DeyBack Wednesday, August 24, 2016 11:56 PM
    Wednesday, August 24, 2016 11:56 PM

All replies

  • code...

    it does http/https redirection based on a database of urls...i have the https 
    part commented out for now, until i get this fixed..

    my classify function...basically, when a new flow is made, i make sure the HTTP 
    header is completed...then I cloned the packet and block, and it gets reinjected 
    inside of the worker thread...

    when data is inbound, i just block and reinject it inline...

    i have commented out the references of NBL's to see if it crashed with or without those...not sure if its needed...i works successfully on 1000s of request, and then BSOD...

    void InjectClonedPacket(UINT64 flowId, UINT16 layerId, UINT32 streamFlags, UINT32 calloutId, NET_BUFFER_LIST *clonedNetBufferList, size_t sizeDataToInject, int setFin) {
    NTSTATUS ntStatus;

    DebugTrace("%s Enter", __FUNCTION__);

    KIRQL irql;
    KeAcquireSpinLock(&SpinLockHttpClassify, &irql);
    ntStatus = FwpsStreamInjectAsync(gInjectionHandle, NULL, 0, flowId, calloutId, layerId, streamFlags,
     clonedNetBufferList, sizeDataToInject, PacketClonedInjectionCompletion, NULL);

    KeReleaseSpinLock(&SpinLockHttpClassify, irql);
    if (ntStatus != STATUS_SUCCESS) {
    FwpsFreeNetBufferList(clonedNetBufferList);
    }
    DebugTrace("%s exit", __FUNCTION__);
    }

    #if(NTDDI_VERSION < NTDDI_WIN7)

    VOID NTAPI
    ClassifyFn(
    IN const FWPS_INCOMING_VALUES  *inFixedValues,
    IN const FWPS_INCOMING_METADATA_VALUES  *inMetaValues,
    IN OUT VOID  *layerData,
    IN const FWPS_FILTER  *filter,
    IN UINT64  flowContext,
    IN OUT FWPS_CLASSIFY_OUT  *classifyOut
    )

    #elseif(NTDDI_VERSION == NTDDI_WIN7)

    void NTAPI ClassifyFn(
    _In_     const FWPS_INCOMING_VALUES0          *inFixedValues,
    _In_     const FWPS_INCOMING_METADATA_VALUES0 *inMetaValues,
    _Inout_        void                           *layerData,
    _In_opt_ const void                           *classifyContext,
    _In_     const FWPS_FILTER1                   *filter,
    _In_           UINT64                         flowContext,
    _Inout_        FWPS_CLASSIFY_OUT0             *classifyOut
    )

    #else

    void NTAPI ClassifyFn(
    _In_        const FWPS_INCOMING_VALUES0          *inFixedValues,
    _In_        const FWPS_INCOMING_METADATA_VALUES0 *inMetaValues,
    _Inout_opt_       void                           *layerData,
    _In_opt_    const void                           *classifyContext,
    _In_        const FWPS_FILTER2                   *filter,
    _In_              UINT64                         flowContext,
    _Inout_           FWPS_CLASSIFY_OUT0             *classifyOut
    )

    #endif

    {
    UNREFERENCED_PARAMETER(inFixedValues);
    UNREFERENCED_PARAMETER(filter);
    UNREFERENCED_PARAMETER(flowContext);
    UNREFERENCED_PARAMETER(classifyOut);

    //PAGED_CODE();

    //DebugTrace("%s Entry", __FUNCTION__);
    classifyOut->actionType = FWP_ACTION_PERMIT;

    FWPS_STREAM_CALLOUT_IO_PACKET *calloutPacket = (FWPS_STREAM_CALLOUT_IO_PACKET*)layerData;
    calloutPacket->streamAction = FWPS_STREAM_ACTION_NONE;
    calloutPacket->countBytesRequired = 0;
    calloutPacket->countBytesEnforced = calloutPacket->streamData->dataLength;

    if (IsDriverUnloading) {
    if (!flowContext) {
    calloutPacket->streamAction = FWPS_STREAM_ACTION_ALLOW_CONNECTION;
    return;
    }

    NET_BUFFER_LIST *clonedNetBufferList = NULL;
    NTSTATUS s = FwpsCloneStreamData(calloutPacket->streamData, NULL, NULL, 0, &clonedNetBufferList);
    if (s != STATUS_SUCCESS) {
    DebugTrace("FwpsAllocateCloneNetBufferList failed");
    return;
    }

    DbgPrintEx(DPFLTR_IHVDRIVER_ID, DPFLTR_ERROR_LEVEL, "injecting data with flow id[%ld] layer id[%d] length[%ld]\r\n", inMetaValues->flowHandle, inFixedValues->layerId,
    calloutPacket->streamData->dataLength);

    InjectClonedPacket(inMetaValues->flowHandle, inFixedValues->layerId, calloutPacket->streamData->flags, CalloutId, clonedNetBufferList, calloutPacket->streamData->dataLength, 0);

    classifyOut->actionType = FWP_ACTION_BLOCK;
    classifyOut->flags = FWPS_CLASSIFY_OUT_FLAG_ABSORB;

    return;
    }

    if ((classifyOut->rights & FWPS_RIGHT_ACTION_WRITE) == 0) {
    //DebugTrace("%s Exit", __FUNCTION__);
    calloutPacket->streamAction = FWPS_STREAM_ACTION_ALLOW_CONNECTION;
    return;
    }

    if (!calloutPacket->streamData->dataLength) {
    calloutPacket->streamAction = FWPS_STREAM_ACTION_ALLOW_CONNECTION;
    return;
    }

    PFLOWPACKET pFlowPacket = NULL;
    if (!flowContext) {
    pFlowPacket = (PFLOWPACKET)ExAllocateFromNPagedLookasideList(&HttpFlowLookList);
    if (!pFlowPacket) {
    calloutPacket->streamAction = FWPS_STREAM_ACTION_ALLOW_CONNECTION;
    return;
    }
    pFlowPacket->flowId = inMetaValues->flowHandle;
    pFlowPacket->stopThread = 0;
    pFlowPacket->flowDeleted = 0;
    pFlowPacket->permitPacket = 0;
    pFlowPacket->allowPacket = 0;
    pFlowPacket->queueInjectionPackets = CQueueCreate();
    pFlowPacket->hPkThread = NULL;
    pFlowPacket->foundHeader = 0;


    //MHashMapInsert(PHashMapHTTPFlows, pFlowPacket->flowId, (void*)pFlowPacket);

    NTSTATUS ntStatus = FwpsFlowAssociateContext(pFlowPacket->flowId, FWPS_LAYER_STREAM_V4, CalloutId, (UINT64)pFlowPacket);
    if (ntStatus != STATUS_SUCCESS) {
    DebugTrace("FwpsFlowAssociateContext failed for HTTP.");

    //MHashMapRemoveByKey(PHashMapHTTPFlows, pFlowPacket->flowId);
    ExFreeToNPagedLookasideList(&HttpFlowLookList, pFlowPacket);
    calloutPacket->streamAction = FWPS_STREAM_ACTION_ALLOW_CONNECTION;
    return;
    }

    else {
    PFLOWTHREADPACKET p = (PFLOWTHREADPACKET)ExAllocatePoolWithTag(NonPagedPool, sizeof(FLOWTHREADPACKET), POOLTAGDRIVER);
    p->flowId = inMetaValues->flowHandle;
    p->queueInjectionPackets = pFlowPacket->queueInjectionPackets;
    p->stopThread = 0;
    CQueueAddRef(pFlowPacket->queueInjectionPackets);

    CQueuePush(QueueHTTPFlowPackets, (void*)p);
    KeSetEvent(&KEventNewHTTPFlow, 0, FALSE);

    pFlowPacket->threadPacket = p;
    }
    }
    else {
    //DebugTrace("got flow context...");
    pFlowPacket = (PFLOWPACKET)flowContext;
    }

    if (pFlowPacket->permitPacket) {
    return;
    }

    if (pFlowPacket->foundHeader) {
    //DebugTrace("header found for[%s]...", pFlowPacket->currentUrl);



    //DbgPrintEx(DPFLTR_IHVDRIVER_ID, DPFLTR_ERROR_LEVEL, "injecting data with flow id[%ld] layer id[%d] length[%ld]\r\n", inMetaValues->flowHandle, inFixedValues->layerId,
    //calloutPacket->streamData->dataLength);

    /*
    for (NET_BUFFER_LIST* pCurrentNBL = calloutPacket->streamData->netBufferListChain; pCurrentNBL;) {
    NET_BUFFER_LIST* pNextNBL = NET_BUFFER_LIST_NEXT_NBL(pCurrentNBL);

    DbgPrintEx(DPFLTR_IHVDRIVER_ID, DPFLTR_ERROR_LEVEL, "FwpsReferenceNetBufferList flow id[%ld]\r\n", pFlowPacket->flowId);
    FwpsReferenceNetBufferList(pCurrentNBL, FALSE);

    pCurrentNBL = pNextNBL;
    }*/

    NET_BUFFER_LIST *clonedNetBufferList = NULL;
    NTSTATUS s = FwpsCloneStreamData(calloutPacket->streamData, NULL, NULL, 0, &clonedNetBufferList);
    if (s != STATUS_SUCCESS) {
    DebugTrace("FwpsCloneStreamData failed");
    DbgPrintEx(DPFLTR_IHVDRIVER_ID, DPFLTR_ERROR_LEVEL, "ERROR fwpsclonestreamdata---header found-- flow id[%ld]\r\n", pFlowPacket->flowId);
    pFlowPacket->threadPacket->stopThread = 1;
    calloutPacket->streamAction = FWPS_STREAM_ACTION_ALLOW_CONNECTION;
    return;
    }

    InjectClonedPacket(inMetaValues->flowHandle, inFixedValues->layerId, calloutPacket->streamData->flags, CalloutId, clonedNetBufferList, calloutPacket->streamData->dataLength, 0);

    classifyOut->actionType = FWP_ACTION_BLOCK;
    //classifyOut->flags |= FWPS_CLASSIFY_OUT_FLAG_ABSORB;
    //classifyOut->rights &= ~(UINT32)FWPS_RIGHT_ACTION_WRITE;

    DebugTrace("%s Exit", __FUNCTION__);
    return;

    }

    char* stream = (char*)ExAllocatePoolWithTag(NonPagedPool, calloutPacket->streamData->dataLength, POOL_TAG_CALLOUT_STREAM);
    if (!stream) {
    DbgPrintEx(DPFLTR_IHVDRIVER_ID, DPFLTR_ERROR_LEVEL, "ERROR exalloate stream failed flow id[%ld]\r\n", pFlowPacket->flowId);
    pFlowPacket->threadPacket->stopThread = 1;
    calloutPacket->streamAction = FWPS_STREAM_ACTION_ALLOW_CONNECTION;
    return;
    }

    SIZE_T streamBytesCopied = 0;
    FwpsCopyStreamDataToBuffer(calloutPacket->streamData, stream, calloutPacket->streamData->dataLength, &streamBytesCopied);
    if (streamBytesCopied != calloutPacket->streamData->dataLength) {
    DebugTrace("streamBytesCopied != streamLength");
    DbgPrintEx(DPFLTR_IHVDRIVER_ID, DPFLTR_ERROR_LEVEL, "ERROR streambytes != streamlength flow id[%ld]\r\n", pFlowPacket->flowId);
    ExFreePoolWithTag(stream, POOL_TAG_CALLOUT_STREAM);
    pFlowPacket->threadPacket->stopThread = 1;
    calloutPacket->streamAction = FWPS_STREAM_ACTION_ALLOW_CONNECTION;
    return;
    }

    if (!(stream[0] == 'G' && stream[1] == 'E' && stream[2] == 'T')) {
    pFlowPacket->permitPacket = 1;
    //KeSetEvent(&pFlowPacket->threadPacket->eventQueue, 0, FALSE);
    ExFreePoolWithTag(stream, POOL_TAG_CALLOUT_STREAM);
    DebugTrace("GET not found...");
    DbgPrintEx(DPFLTR_IHVDRIVER_ID, DPFLTR_ERROR_LEVEL, "ERROR get not found flow id[%ld]\r\n", pFlowPacket->flowId);
    pFlowPacket->threadPacket->stopThread = 1;
    calloutPacket->streamAction = FWPS_STREAM_ACTION_ALLOW_CONNECTION;
    return;
    }

    for (int loopa = ((int)streamBytesCopied)-1; loopa >= 0; loopa--) {
    if ((loopa - 3) < 0) {
    DebugTrace("need more data...");
    DbgPrintEx(DPFLTR_IHVDRIVER_ID, DPFLTR_ERROR_LEVEL, "ERROR need more data flow id[%ld]\r\n", pFlowPacket->flowId);
    calloutPacket->streamAction = FWPS_STREAM_ACTION_NEED_MORE_DATA;
    calloutPacket->countBytesRequired = (UINT32)streamBytesCopied + 1;
    classifyOut->actionType = FWP_ACTION_NONE;
    ExFreePoolWithTag(stream, POOL_TAG_CALLOUT_STREAM);
    return;
    }

    if (stream[loopa] == '\n') {
    if (stream[loopa - 1] == '\r') {
    if (stream[loopa - 2] == '\n') {
    if (stream[loopa - 3] == '\r') {
    pFlowPacket->foundHeader = 1;
    break;
    }
    }
    }
    }
    }

    if (!pFlowPacket->foundHeader) {
    DebugTrace("header not found, need more data");
    DbgPrintEx(DPFLTR_IHVDRIVER_ID, DPFLTR_ERROR_LEVEL, "ERROR header not found, need more data flow id[%ld]\r\n", pFlowPacket->flowId);
    calloutPacket->streamAction = FWPS_STREAM_ACTION_NEED_MORE_DATA;
    classifyOut->actionType = FWP_ACTION_NONE;
    calloutPacket->countBytesRequired = (UINT32)streamBytesCopied + 1;
    ExFreePoolWithTag(stream, POOL_TAG_CALLOUT_STREAM);
    return;
    }

    /*
    for (NET_BUFFER_LIST* pCurrentNBL = calloutPacket->streamData->netBufferListChain; pCurrentNBL;) {
    NET_BUFFER_LIST* pNextNBL = NET_BUFFER_LIST_NEXT_NBL(pCurrentNBL);

    DbgPrintEx(DPFLTR_IHVDRIVER_ID, DPFLTR_ERROR_LEVEL, "FwpsReferenceNetBufferList flow id[%ld]\r\n", pFlowPacket->flowId);
    FwpsReferenceNetBufferList(pCurrentNBL, FALSE);

    pCurrentNBL = pNextNBL;
    }*/

    NET_BUFFER_LIST *clonedNetBufferList = NULL;
    NTSTATUS s = FwpsCloneStreamData(calloutPacket->streamData, NULL, NULL, 0, &clonedNetBufferList);
    if (s != STATUS_SUCCESS) {
    DbgPrintEx(DPFLTR_IHVDRIVER_ID, DPFLTR_ERROR_LEVEL, "ERROR ffwpsclonestreamdata failed flow id[%ld]\r\n", pFlowPacket->flowId);
    ExFreePoolWithTag(stream, POOL_TAG_CALLOUT_STREAM);
    pFlowPacket->threadPacket->stopThread = 1;
    calloutPacket->streamAction = FWPS_STREAM_ACTION_ALLOW_CONNECTION;
    return;
    }

    PINJECTION_PACKET pInjectionPacket = (PINJECTION_PACKET)ExAllocatePoolWithTag(NonPagedPool, sizeof(INJECTION_PACKET), 'eee');
    if (!pInjectionPacket) {
    DbgPrintEx(DPFLTR_IHVDRIVER_ID, DPFLTR_ERROR_LEVEL, "ERROR exallocate failed for injection packet flow id[%ld]\r\n", pFlowPacket->flowId);
    ExFreePoolWithTag(stream, POOL_TAG_CALLOUT_STREAM);
    pFlowPacket->threadPacket->stopThread = 1;
    calloutPacket->streamAction = FWPS_STREAM_ACTION_ALLOW_CONNECTION;
    return;
    }
    pInjectionPacket->data = stream;
    pInjectionPacket->data_length = calloutPacket->streamData->dataLength;
    pInjectionPacket->flow_id = inMetaValues->flowHandle;
    pInjectionPacket->layerId = inFixedValues->layerId;
    pInjectionPacket->clonedNetBufferList = clonedNetBufferList;
    pInjectionPacket->streamFlags = calloutPacket->streamData->flags;
    CQueuePush(pFlowPacket->queueInjectionPackets, (void*)pInjectionPacket);

    DbgPrintEx(DPFLTR_IHVDRIVER_ID, DPFLTR_ERROR_LEVEL, "queueing data with flow id[%ld] layer id[%d] length[%ld]\r\n", inMetaValues->flowHandle, inFixedValues->layerId,
    calloutPacket->streamData->dataLength);
    KeSetEvent(&(pFlowPacket->queueInjectionPackets->eventQueue), 0, FALSE);

    classifyOut->actionType = FWP_ACTION_BLOCK;
    //classifyOut->flags |= FWPS_CLASSIFY_OUT_FLAG_ABSORB;
    //classifyOut->rights &= ~(UINT32)FWPS_RIGHT_ACTION_WRITE;

    //DebugTrace("%s Exit", __FUNCTION__);
    }

    Wednesday, August 24, 2016 5:39 AM
  • oh...I believe I fixed the problem..

    so I was returning FWPS_STREAM_ACTION_NEED_MORE_DATA ---and not checking if the data length changed or remained constant..so I guess it went into an infinite loop and crashed the stream

    • Marked as answer by DeyBack Wednesday, August 24, 2016 11:56 PM
    Wednesday, August 24, 2016 11:56 PM
  • well...still crashing, after several 100 successful attempts...definitely a bug on WFP

    kd> !analyze -v
    *******************************************************************************
    *                                                                             *
    *                        Bugcheck Analysis                                    *
    *                                                                             *
    *******************************************************************************

    DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
    An attempt was made to access a pageable (or completely invalid) address at an
    interrupt request level (IRQL) that is too high.  This is usually
    caused by drivers using improper addresses.
    If kernel debugger is available get stack backtrace.
    Arguments:
    Arg1: 000000000000003c, memory referenced
    Arg2: 0000000000000002, IRQL
    Arg3: 0000000000000001, value 0 = read operation, 1 = write operation
    Arg4: fffff800eca53192, address which referenced memory

    Debugging Details:
    ------------------


    BUGCHECK_P1: 3c

    BUGCHECK_P2: 2

    BUGCHECK_P3: 1

    BUGCHECK_P4: fffff800eca53192

    WRITE_ADDRESS:  000000000000003c 

    CURRENT_IRQL:  2

    FAULTING_IP: 
    tcpip!TcpBeginTcbSend+732
    fffff800`eca53192 f0ff403c        lock inc dword ptr [rax+3Ch]

    CPU_COUNT: 1

    CPU_MHZ: 6a0

    CPU_VENDOR:  GenuineIntel

    CPU_FAMILY: 6

    CPU_MODEL: 45

    CPU_STEPPING: 1

    DEFAULT_BUCKET_ID:  WIN8_DRIVER_FAULT

    BUGCHECK_STR:  AV

    PROCESS_NAME:  System

    ANALYSIS_VERSION: 10.0.10240.9 amd64fre

    DPC_STACK_BASE:  FFFFF80076370FB0

    TRAP_FRAME:  fffff8007636f800 -- (.trap 0xfffff8007636f800)
    NOTE: The trap frame does not contain all registers.
    Some register values may be zeroed or incorrect.
    rax=0000000000000000 rbx=0000000000000000 rcx=ffffcf81da480ea0
    rdx=0000000000000000 rsi=0000000000000000 rdi=0000000000000000
    rip=fffff800eca53192 rsp=fffff8007636f990 rbp=fffff8007636fa90
     r8=ffffcf81da480e90  r9=ffffcf81da480dd0 r10=ffffcf81da480dd0
    r11=ffffcf81da480fe8 r12=0000000000000000 r13=0000000000000000
    r14=0000000000000000 r15=0000000000000000
    iopl=0         nv up ei pl nz na pe nc
    tcpip!TcpBeginTcbSend+0x732:
    fffff800`eca53192 f0ff403c        lock inc dword ptr [rax+3Ch] ds:00000000`0000003c=????????
    Resetting default scope

    LAST_CONTROL_TRANSFER:  from fffff80074a6898e to fffff800749dfe90

    STACK_TEXT:  
    fffff800`7636ef08 fffff800`74a6898e : 00000000`00000000 00000000`00000000 fffff800`7636f070 fffff800`7495f7a4 : nt!DbgBreakPointWithStatus
    fffff800`7636ef10 fffff800`74a6829f : 00000000`00000003 fffff800`7636f070 fffff800`749e7290 00000000`000000d1 : nt!KiBugCheckDebugBreak+0x12
    fffff800`7636ef70 fffff800`749d93a4 : fffff800`7636fa48 00000000`0006cf46 00000000`00000000 fffff800`749233e7 : nt!KeBugCheck2+0x8ab
    fffff800`7636f680 fffff800`749e4de9 : 00000000`0000000a 00000000`0000003c 00000000`00000002 00000000`00000001 : nt!KeBugCheckEx+0x104
    fffff800`7636f6c0 fffff800`749e363a : 00000000`00000001 fffff800`7636fa48 fffff800`7636b000 fffff800`76371000 : nt!KiBugCheckDispatch+0x69
    fffff800`7636f800 fffff800`eca53192 : 00000000`fffffffe ffffe000`3c98cd10 ffff7a41`47f1303e ffffe000`377e5490 : nt!KiPageFault+0x23a
    fffff800`7636f990 fffff800`eca51e52 : 00000000`00020000 fffff800`ebf27ce7 ffffe000`37d19008 00000000`00000001 : tcpip!TcpBeginTcbSend+0x732
    fffff800`7636fc70 fffff800`eca744ab : 00000000`00000001 00000000`00000001 ffffe000`38183010 ffffcf81`ce31adf0 : tcpip!TcpTcbSend+0x226
    fffff800`7636ffc0 fffff800`eca4991c : ffffd001`0a1fa82c 00000000`00021b67 00000000`00000000 00000000`00000000 : tcpip!TcpFlushDelay+0x20a
    fffff800`76370070 fffff800`eca43423 : ffffe000`37d1dbb0 00000000`00005000 00000000`000033c6 fffff800`74b433c6 : tcpip!TcpPreValidatedReceive+0x3cc
    fffff800`76370170 fffff800`eca76e32 : ffffe000`37fc63d0 fffff800`76370600 00000000`00000006 ffffcf81`da270006 : tcpip!IpFlcReceivePreValidatedPackets+0x649
    fffff800`76370350 fffff800`74932fc3 : 00000000`00000003 00000000`00000000 ffffe000`37d2eb10 fffff800`7636b000 : tcpip!FlReceiveNetBufferListChainCalloutRoutine+0x102
    fffff800`76370480 fffff800`eca77076 : fffff800`eca76d30 fffff800`763705a0 00000000`00000010 00000000`00000801 : nt!KeExpandKernelStackAndCalloutInternal+0xf3
    fffff800`76370570 fffff800`ec887a53 : 00000000`00000000 fffff800`76370651 00000000`00000003 fffff800`eca54550 : tcpip!FlReceiveNetBufferListChain+0xb6
    fffff800`763705f0 fffff800`ec887e7f : 00000000`00000001 00000000`00000000 00000000`00000000 00000000`00000003 : ndis!ndisMIndicateNetBufferListsToOpen+0x123
    fffff800`763706b0 fffff800`ec8886b2 : ffffe000`380e91a0 fffff800`74b37201 fffff800`ec894540 00000000`00000000 : ndis!ndisMTopReceiveNetBufferLists+0x22f
    fffff800`76370740 fffff800`ed7861c4 : ffffe000`38115000 fffff800`ed786efc ffffe000`38115e00 ffffcf81`ce0b2df0 : ndis!NdisMIndicateReceiveNetBufferLists+0x732
    fffff800`76370930 fffff800`ed786a9d : 00000000`00000001 ffffcf81`cded2df0 ffffe000`38115000 00000000`00000003 : e1i63x64!RECEIVE::RxIndicateNBLs+0xd4
    fffff800`76370970 fffff800`ed779150 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : e1i63x64!RECEIVE::RxProcessInterrupts+0x19d
    fffff800`763709f0 fffff800`ed77957e : ffffcf81`cda4eff0 ffffe000`38115000 ffff0001`00000000 ffff0001`00000000 : e1i63x64!INTERRUPT::MsgIntDpcTxRxProcessing+0x1c0
    fffff800`76370a60 fffff800`ed778b78 : fffff800`76370b79 ffff0001`00000000 00000000`00000000 ffffe000`380e91a0 : e1i63x64!INTERRUPT::MsgIntMessageInterruptDPC+0x13e
    fffff800`76370ac0 fffff800`ec889e12 : 00000000`00000000 fffff800`ed0b3d08 ffffe000`380c3502 fffff800`748c3e17 : e1i63x64!INTERRUPT::MiniportMessageInterruptDPC+0x28
    fffff800`76370b00 fffff800`748c3910 : 00000000`00000000 fffff800`7488b000 fffff800`74b3f480 ffffe000`37cf0f44 : ndis!ndisInterruptDpc+0x1a3
    fffff800`76370be0 fffff800`748c2c57 : 00000000`00000000 ffffe000`3bd81080 fffff800`74b88180 00000000`00000000 : nt!KiExecuteAllDpcs+0x1b0
    fffff800`76370d30 fffff800`749dc3d5 : 00000000`00000000 fffff800`74b88180 00000000`00000000 ffffe000`3c98cd10 : nt!KiRetireDpcList+0xd7
    fffff800`76370fb0 fffff800`749dc1d9 : 00001f80`00000000 00000000`00000010 00000000`00000000 00000000`00000000 : nt!KxRetireDpcList+0x5
    ffffd001`0aeaf3f0 fffff800`749de2fa : 00000000`00000001 ffffe000`38cbf140 ffffe000`38204410 00000000`00000000 : nt!KiDispatchInterruptContinue
    ffffd001`0aeaf420 fffff800`748be652 : ffffe000`3c98cd80 ffffe000`3c98cd10 ffffe000`3ca04440 ffffd001`0aeaf6e0 : nt!KiDpcInterrupt+0xca
    ffffd001`0aeaf5b0 fffff800`eca5222c : ffffd001`0aeaf600 00000000`00000001 00000000`00000000 00000000`00000b9c : nt!KeReleaseSpinLock+0x22
    ffffd001`0aeaf5e0 fffff800`eca47bc5 : ffffe000`3c98cd10 00000000`00000000 ffffd001`0aeaf9c2 ffffe000`3c98cd10 : tcpip!TcpTcbSend+0x600
    ffffd001`0aeaf930 fffff800`eca4780c : 00000000`00021b67 00000000`00001000 00000000`00000001 ffffd001`0aeafad0 : tcpip!TcpEnqueueTcbSendOlmNotifySendComplete+0xa5
    ffffd001`0aeaf960 fffff800`ec55e6a6 : 00000000`00000000 00000000`00707464 ffffcf81`da126df0 00000000`00000122 : tcpip!TcpEnqueueTcbSend+0x2ac
    ffffd001`0aeafa60 fffff800`ec55ec77 : ffffe000`37aa5270 ffffe000`3c98cd10 00000000`00000000 fffff800`edca0000 : NETIO!StreamInjectRequestsToStack+0x24a
    ffffd001`0aeafb40 fffff800`ec55ed82 : ffffe000`37aa5270 ffffe000`36d62130 00000000`00000002 ffffcf81`da18eff8 : NETIO!StreamPermitDataHelper+0x5f
    ffffd001`0aeafb70 fffff800`7493f304 : ffffcf81`da41afb0 fffff800`748c6b00 00000000`00000000 fffff800`ec55ecc0 : NETIO!StreamPermitRemoveDataWorkerRoutine+0xc2
    ffffd001`0aeafbe0 fffff800`7493fa2f : 00000000`00000000 fffff800`7493f284 ffffe000`39747700 00000000`00000000 : nt!IopProcessWorkItem+0x80
    ffffd001`0aeafc50 fffff800`74985c10 : ffffe000`35d33880 ffffe000`39747700 00000000`00000080 ffffe000`39747700 : nt!ExpWorkerThread+0x69f
    ffffd001`0aeafd00 fffff800`749df8c6 : fffff800`74b88180 ffffe000`39747700 ffffe000`35d33880 00000000`00000000 : nt!PspSystemThreadStartup+0x58
    ffffd001`0aeafd60 00000000`00000000 : ffffd001`0aeb0000 ffffd001`0aeaa000 00000000`00000000 00000000`00000000 : nt!KiStartSystemThread+0x16


    STACK_COMMAND:  kb

    FOLLOWUP_IP: 
    e1i63x64!RECEIVE::RxIndicateNBLs+d4
    fffff800`ed7861c4 40f6c702        test    dil,2

    SYMBOL_STACK_INDEX:  11

    SYMBOL_NAME:  e1i63x64!RECEIVE::RxIndicateNBLs+d4

    FOLLOWUP_NAME:  MachineOwner

    MODULE_NAME: e1i63x64

    IMAGE_NAME:  e1i63x64.sys

    DEBUG_FLR_IMAGE_TIMESTAMP:  51496739

    BUCKET_ID_FUNC_OFFSET:  d4

    FAILURE_BUCKET_ID:  AV_VRF_e1i63x64!RECEIVE::RxIndicateNBLs

    BUCKET_ID:  AV_VRF_e1i63x64!RECEIVE::RxIndicateNBLs

    PRIMARY_PROBLEM_CLASS:  AV_VRF_e1i63x64!RECEIVE::RxIndicateNBLs

    ANALYSIS_SOURCE:  KM

    FAILURE_ID_HASH_STRING:  km:av_vrf_e1i63x64!receive::rxindicatenbls

    FAILURE_ID_HASH:  {da165245-bf6e-90ea-00d7-b47bffc449f5}

    Followup:     MachineOwner
    ---------

    Thursday, August 25, 2016 9:12 AM