locked
NTSTATUS_DATA_NOT_ACCEPTED when adding IP Options to IP Header (includes code) RRS feed

  • Question

  • After working on my problem for a while, I thought I hit upon a solution for adding IP options into the IP header.  Here is the code that is contained in a WPF callout helper function.  Packets are queued to the helper function from my OUTBOUND_TRANSPORT_V4 classification function, with flow information and such included in the "packet" structure.  I've used DebugPrint to verify that the ip packets generated are valid, and even exported them into wireshark to ensure they get parsed properly. All packets submitted to this function have been cloned in the classifyFn, and have been appropriately referenced.

    Here's the problem - even packet injected by the function returns status NTSTATUS_DATA_NOT_ACCEPTED (0xC000021B) in the completion function.  No data appears on the wire.  If I remove the portion of this function that performs the NDIS_REWIND and IP option insertion, packets flow just fine, with no errors (NTSTATUS_SUCCESS in completion function).

    Based on other posts here, I suspect there's an offset problem, but I have no idea how to deal with it. anyone have an idea what may be causing the problem?  Changing the Injection function from TRANSPORT to NETWORK does not fix the issue.  Are there any other suggestions?

    void
    TaggingHelper(
       _In_ void* StartContext
       )
    /* ++
    
       This worker thread waits for the packet queue event when the queue is
       empty; and it will be woken up when there are packets queued needing to 
       be tagged with username information. Once awaking, It will run in a loop
       to modify-reinject packets until the packet queue is exhausted
       (and it will go to sleep waiting for more work).
    
       The worker thread will end once it detected the driver is unloading.
    
    -- */
    {
       TAGGING_PENDED_PACKET* packet;
       LIST_ENTRY* listEntry;
       KLOCK_QUEUE_HANDLE packetQueueLockHandle;
       NTSTATUS status;
       UCHAR *tmpBuffer = NULL;
       UCHAR *bufferPointer = NULL;
       UINT8 versionIHL = 0;
       UINT16 totalLength = 0;
       UINT8 packetHeaderWords = 0;
       UINT8 optionStringLength = 0;
       UCHAR *optionString = NULL;
       UINT8 headerLength = 0;
       UINT16 checksum = 0;
       INT i = 0;
       UCHAR *abuffer = NULL;
       WSACMSGHDR ipOption;
       WSACMSGHDR *enhancedControlData = NULL;
       UINT enhancedControlDataLength  = 0;
    
       UNREFERENCED_PARAMETER(StartContext);
       DbgPrintEx(DPFLTR_IHVNETWORK_ID,DPFLTR_INFO_LEVEL,"Starting Up Helper Thread\n");
    
       for(;;)
       {
          KeWaitForSingleObject(
             &gPacketQueueEvent,
             Executive, 
             KernelMode, 
             FALSE, 
             NULL
             );
    
          if (gDriverUnloading)
          {
             break;
          }
    
          NT_ASSERT(!IsListEmpty(&gPacketQueue));
    
          KeAcquireInStackQueuedSpinLock(
             &gPacketQueueLock,
             &packetQueueLockHandle
             );
    
    		listEntry = RemoveHeadList(&gPacketQueue);
    
    		KeReleaseInStackQueuedSpinLock(&packetQueueLockHandle);
    
    		packet = CONTAINING_RECORD(
    						listEntry,
    						TAGGING_PENDED_PACKET,
    						listEntry
    						);
       	   DbgPrintEx(DPFLTR_IHVNETWORK_ID,DPFLTR_INFO_LEVEL,"Processing Packet.\n");
    
    		//create the option string.
    		CreateOptionString(&optionString,&optionStringLength,0xdeadbeef,0xdeadbeef);
    
    		//		DebugPrintByteString(optionString,optionStringLength,0x04);
    
    		//add the IP header to the net buffer list.
    		status = FwpsConstructIpHeaderForTransportPacket0(packet->netBufferList,
    															packet->ipHeaderSize,
    															AF_INET,
    															packet->localAddr,
    															packet->remoteAddr,
    															(IPPROTO) packet->protocol,
    															packet->endpointHandle,
    															packet->controlData,
    															packet->controlDataLength,
    															0,
    															NULL,
    															packet->interfaceIndex,
    															packet->subInterfaceIndex);
    		if (!NT_SUCCESS(status))
    		{
    			DbgPrintEx(DPFLTR_IHVNETWORK_ID,DPFLTR_ERROR_LEVEL,"Failed to create IP header, return status was %x.\n",status);
    			//if we can't backup the option header distance in the inbound net buffer list, block the packet.
    			goto Exit;
    		}
    
    		//rewind the nbl to fit the option header and IP header.  This apparently creates contiguous space.
    		bufferPointer = NULL;
    		status = NdisRetreatNetBufferDataStart(NET_BUFFER_LIST_FIRST_NB(packet->netBufferList),optionStringLength,0,NULL);
    		if (!NT_SUCCESS(status))
    		{
    			DbgPrintEx(DPFLTR_IHVNETWORK_ID,DPFLTR_ERROR_LEVEL,"Failed to back up the option header.\n");
    			//if we can't backup the option header distance in the inbound net buffer list, block the packet.
    			goto Exit;
    		}
    
    		headerLength = IP_HEADER_LENGTH;
    		//Get a pointer to the beginning of the NBL data.
    		bufferPointer = (UCHAR *)NdisGetDataBuffer(NET_BUFFER_LIST_FIRST_NB(packet->netBufferList),IP_HEADER_LENGTH + optionStringLength,NULL,1,0);
    		if (bufferPointer == NULL)
    		{
    			DbgPrintEx(DPFLTR_IHVNETWORK_ID,DPFLTR_ERROR_LEVEL,"could not get buffer pointer to packet.\n");
    			goto Exit;
    		}
    
    		//allocate temporary buffer
    		tmpBuffer = (UCHAR *)ExAllocatePoolWithTag(NonPagedPool,headerLength, IP_TAGGING_POOL_TAG);
    		if (tmpBuffer == NULL)
    		{
    			DbgPrintEx(DPFLTR_IHVNETWORK_ID,DPFLTR_ERROR_LEVEL,"Out of memory getting tmpBuffer.\n");
    			goto Exit;
    		}
    
    		//copy the original header to a temporary buffer.
    		RtlCopyMemory(tmpBuffer,bufferPointer + optionStringLength,headerLength);
    		
    		//copy the IP header to tbe beginning of the data space.
    		RtlCopyMemory(bufferPointer,tmpBuffer,headerLength);
    
    		//now add in the option string
    		RtlCopyMemory(bufferPointer + headerLength, optionString, optionStringLength);
    
    		//change the packet fields
    
    		totalLength = RtlUshortByteSwap(*((UINT16 *)(bufferPointer + 2)));
    		versionIHL = ((headerLength + optionStringLength) / 4) | 0x40;
    		totalLength += optionStringLength;
    		*(UINT8 *)bufferPointer = versionIHL;
    		*((UINT16 *)(bufferPointer + 2)) = RtlUshortByteSwap(totalLength);
    		*((UINT16 *)(bufferPointer + 10)) = 0x0000;  //reset checksum	
    
    		//Try to use FwpsconstructIpHeader to fix checksum.
    		status = FwpsConstructIpHeaderForTransportPacket0(packet->netBufferList,
    															headerLength + optionStringLength,
    															AF_INET,
    															packet->localAddr,
    															packet->remoteAddr,
    															(IPPROTO) packet->protocol,
    															packet->endpointHandle,
    															packet->controlData,
    															packet->controlDataLength,
    															0,
    															NULL,
    															packet->interfaceIndex,
    															packet->subInterfaceIndex);
    		if (!NT_SUCCESS(status))
    		{
    			DbgPrintEx(DPFLTR_IHVNETWORK_ID,DPFLTR_ERROR_LEVEL,"Failed to validate IP header, return status was %x.\n",status);
    			//if we can't backup the option header distance in the inbound net buffer list, block the packet.
    			goto Exit;
    		}
    
    		//packet modification done.  Reinject.
    		//print ICMP packets to debug for check	
    /*		if (packet->protocol == IPPROTO_ICMP)
    		{
    
    			abuffer = (UCHAR *)ExAllocatePoolWithTag(NonPagedPool,totalLength,IP_TAGGING_POOL_TAG);
    			bufferPointer = (UCHAR *)NdisGetDataBuffer(NET_BUFFER_LIST_FIRST_NB(packet->netBufferList),totalLength,abuffer,1,0);
    			for (i = 0; i < totalLength ; ++i)
    			{
    			   DbgPrintEx(DPFLTR_IHVNETWORK_ID,DPFLTR_INFO_LEVEL,"%.2x ",bufferPointer[i]);
    			   if (((i + 1) % 4) == 0)
    				 DbgPrintEx(DPFLTR_IHVNETWORK_ID,DPFLTR_INFO_LEVEL,"\n");			
    			}
    
    		}
    */
    		//Store the rewind length for later.
    		packet->totalRewind = optionStringLength;
    
    		//packet modification done.  Reinject.
    
    		status = FwpsInjectTransportSendAsync0(
    					gInjectionHandle,
    					NULL,
    					packet->endpointHandle,
    					0,
    					NULL,
    					AF_INET,
    					packet->compartmentId,
    					packet->netBufferList,
    					TaggingHelperInjectComplete,
    					packet);
    
    		if (!NT_SUCCESS(status))
    		{
    			DbgPrintEx(DPFLTR_IHVNETWORK_ID,DPFLTR_INFO_LEVEL,"Packet Inject Failed.\n");
    		}
    		else
    		{
    			DbgPrintEx(DPFLTR_IHVNETWORK_ID,DPFLTR_INFO_LEVEL,"Packet Inject succeeded.\n");
    			packet = NULL;
    		}
    Exit:
    
    		if (packet != NULL)
    		{
    			if (packet->netBufferList != NULL)
    				FwpsFreeCloneNetBufferList0(packet->netBufferList, 0);
    			FreeTaggingPendedPacket(packet);
    		}
    
    		//free the temp buffer
    		if (optionString != NULL)
    		{
    			ExFreePoolWithTag(optionString,IP_TAGGING_POOL_TAG);
    			optionString = NULL;
    			optionStringLength = 0;
    		}
    
    		//reset counting variables
    		bufferPointer = NULL;
    		versionIHL = 0;
    		totalLength = 0;
    		packetHeaderWords = 0;
    
    		KeAcquireInStackQueuedSpinLock(
    			&gPacketQueueLock,
    			&packetQueueLockHandle
    			);
    
    		if (IsListEmpty(&gPacketQueue) && !gDriverUnloading)
    		{
    			KeClearEvent(&gPacketQueueEvent);
    		}
    
    		KeReleaseInStackQueuedSpinLock(&packetQueueLockHandle);
    
       }  //closes the infinite loop
    
       NT_ASSERT(gDriverUnloading);
    
       //
       // Discard all the pended packets if driver is being unloaded.
       //
    
    	KeAcquireInStackQueuedSpinLock(
    		&gPacketQueueLock,
    		&packetQueueLockHandle
    		);
    
    	while (!IsListEmpty(&gPacketQueue))
    	{
    		listEntry = RemoveHeadList(&gPacketQueue);
    
    		packet = CONTAINING_RECORD(
    						listEntry,
    						TAGGING_PENDED_PACKET,
    						listEntry
    						);
    
    		FreeTaggingPendedPacket(packet);
    	}
    
    	KeReleaseInStackQueuedSpinLock(&packetQueueLockHandle);
    	PsTerminateSystemThread(STATUS_SUCCESS);
    
    	return;
    }


    Tuesday, January 22, 2013 11:07 PM

All replies

  • I am trying to write a WFP network filter driver that will add the Timestamp option to almost every packet leaving a system.  Following a lot of searching here, I settled on the following process:

    1)  Filter on FLOW_ESTABLISHED, and send context to the outbound_transport_v4 layer.  This works perfectly.

    2)  The oubound_transport classify function cloned the NBL and blocks/absorbs.  It passes the packet to a worker thread (same as DD_proxy example).

    3)  The worker thread does the following:

    • Calls NdisRetreatNetBufferDataStart on each NB in the NBL, retreating 40 bytes (20 for IP header, 20 for timestamp)
    • Calls NdisAdvanceNetBufferDataStart on each NB in the NBL, advancing 40 bytes, with freeMDL = FALSE 

    According to the forum posts here, that should create 40 bytes of contiguous space to add my IP header.

    • Calls FwpsConstructIpHeaderForTransportPacket0() with headerIncludeHeaderLength = 0.
    • Calls NdisGetDataBuffer on the NBL with a length of 40 bytes, alignment of 1, alignOffset = 0. 

    For some reason, this always results in NdisGetDataBuffer returning a NULL pointer.

    Since the documentation of NdisGetDataBuffer isn't really precise on the topic, how does it determine what pointer to return?

    Is there a better way to allocate the header space in the buffer?  I've tried many ways, but they all results in the completionFn returning a NBL status of STATUS_DATA_NOT_ACCEPTED.

    any ideas?

    thanks!

    -alex

    Friday, January 18, 2013 3:53 PM
  • You can't just retreat the clone.  At OUTBOUND_TRANSPORT, the IPHeader has not been created yet... so when you retreat, you are essentially overwriting other info.  The solution would be to create a new NBL big enough for the current NBL + the IPHeader+the option field.  copy the contents of the current NBL into this, modify your option field, and call FwpsConstructIpHeaderForTransportPacket indicating that you have an IPHeaderInclude (and the size of that header).

    Hope this helps,


    Dusty Harper [MSFT]
    Microsoft Corporation
    ------------------------------------------------------------
    This posting is provided "AS IS", with NO warranties and confers NO rights
    ------------------------------------------------------------

    Thursday, January 24, 2013 4:04 AM
    Moderator
  • Dusty -

    Thank you very much for the response.  just to make sure I get this right, are these the steps you'd recommend?

    1)  Allocate a NBL pool using NdisAllocateNetBufferListPool with the NET_BUFFER_LIST_POOL_PARAMETERS Datasize = 0, fAllocateNetBuffer = TRUE.

    2)  Allocate an MDL using NetAllocateMDL that contains enough space for the original packet, plus IP header, plus my option

    3)  Allocate the NBL and NB using NetAllocateNetBufferAndNetBufferList and the MDL initialized in step 2.

    4)  Copy all transport layer data from the old packet to the new

    5)  Add IP header and Options to the new packet

    6)  Adjust checksums, header lengths, total length, etc, in the new packet.

    7)  Reference the new NBL

    8)  Inject the new packet, free the old (clone) NBL.

    Does that sound about right?

    Thanks!

    Alex

    Thursday, January 24, 2013 6:27 PM