locked
Flow control on outbound stream RRS feed

  • Question

  • Is there any way from stream layer callout to tell local server not sending data too fast?

     

    My callout working on stream layer, filter outgoing data. The work flow is:

    1. My stream callout was called by WFP with data. My callout makes a copy and send to userland, block/discard the original data with FWP_ACTION_BLOCK.

    2. When userland complete processing, it will send modified data to my callout with IoCtrl, my callout will then create a new NBL on the new data and inject it into WFP with FwpsStreamInjectAsync0.

    3. After WFP finished processing, it will call my callout complete routine, the complete routine will release the NBL and data buffer resource.

     

    Everything works fine until I installed my callout on a web server. If there are many clients downloading large files at the same time, the web server will send too much data to my stream callout and my callout will inject too much data into WFP. The problem is, WFP will not complete injection quickly, and eventurally there will be thousands of injections not completed by WFP and used up kernel memory.

     

    Is there any way from my callout to tell the web server not sending data too fast?

    If my callout passby all data, the kernel memory usage will not increase, how the WFP control web server outgoing flow in this case?

    Wednesday, July 16, 2008 11:37 PM

Answers

  • If you return FWP_ACTION_BLOCK but keep the indicated send data referenced (e.g. via FwpsCloneStreamData0 or FwpsReferenceNetBufferList0 on the whole chain), the IRPs associated with the data wouldn't complete -- the behavior should be the same as TDIs.

     

    However as I mentioned, not completing IRPs would not necessarily slow down the send rate of apps -- it depends on how the app was written.

     

    When TCP's buffer is completely filled, winsock's write data calls would start to fail.

     

    Hope this helps,

    Biao.W.

    Tuesday, July 22, 2008 4:34 AM

All replies

  • This is a tough problem with elements outside WFP's control.

     

    To maximize send throughput, many winsocket applications perform overlapped writes to the socket -- meaning they don't wait for the previous write to complete before issueing the subsequent ones.

     

    Winsock perfectly allows such overlapped writes and forwards the data to the TCP/IP stack which does its best to send data into the wire. The TCP/IP stack has a rather big buffering limit such that it can continue to accept data from winsock even if it needs to throttle down the send rate to allow the remote peer to catch up.

     

    In the presenceof WFP stream callouts, TCP/IP would indicate data to the the callouts as agressively as it accepts data from Winsock.

     

    One solution to this problem would be to monitor your kernel memory usage and return error page to the client with friendly message to the effect of "the server is busy, please try later" when it reaches certain threshold.

     

    Hope this helps,

    Biao.W.

     

    Thursday, July 17, 2008 4:52 AM
  • Returning "error page" is not acceptable in my application. How to return "error page" when user in the middle of downloading large file? And what if not web application, for example database application.

     

    How does the TCP/IP stack do when it's "big buffer" eventurally got full? Will it tell winsock applications to slow down, or just keep sending IRP not completed?

     

    And what WFP will do when my callout returned FWP_ACTION_BLOCK? will it just complete the sending IRP with STATUS_SUCCESS? If this is the case, then it explains why the application keep sending data, because it think all sent data was successfully deliverred quickly.

     

    I start missing the old TDI time, in which case my filter just pending the sending IRP not completed, and the application will eventurally slow down sending.

     

    Thursday, July 17, 2008 8:40 PM
  • If you return FWP_ACTION_BLOCK but keep the indicated send data referenced (e.g. via FwpsCloneStreamData0 or FwpsReferenceNetBufferList0 on the whole chain), the IRPs associated with the data wouldn't complete -- the behavior should be the same as TDIs.

     

    However as I mentioned, not completing IRPs would not necessarily slow down the send rate of apps -- it depends on how the app was written.

     

    When TCP's buffer is completely filled, winsock's write data calls would start to fail.

     

    Hope this helps,

    Biao.W.

    Tuesday, July 22, 2008 4:34 AM