none
Socket.BeginSend is not guaranteed to write the whole provided buffer before calling back RRS feed

  • Question

  • Hi everybody,

    First, excuse me if this is not the right forum. System.Net's forum has been retired.

    Some people claim it is completely safe to call Socket.BeginSend concurrently (http://social.msdn.microsoft.com/Forums/en/netfxnetcom/thread/53aa930e-ce51-4993-a23b-040905aed46f).

    Others (more often) claim that explicit synchronization of writers is needed.

    MSDN documentation raises reasonable doubt:

    <Extracted from EndSend's documentation page>

    If you are using a connectionless protocol, EndSend will block until the datagram is sent. If you are using a connection-oriented protocol, EndSend will block until some of the buffer was sent. If the return value from EndSend indicates that the buffer was not completely sent, call the BeginSend method again, modifying the buffer to hold the unsent data.

    This paragraph is important, because it can affect asynchronous send's thread safety. Let me explain:

    If I call BeginSend (concurrently; e.g., let's say twice), the first one ends with a premature callback, and the callback calls BeginSend again to ask for the sending of the remaining bytes, how am I guaranteed that the second original BeginSend is not able to send in between? Otherwise, my transmissions get mixed up.

    I can only come up with these possibilities:

    1. Even if .NET's API allows partial sends, Winsock never produces them.

    2. When the first BeginSend (in my example) is call backed with a partial result, I have the opportunity to call BeginSend again; and only after that, in time order, the second original BeginSend is also called back (reporting 0 sent bytes) and given the opportunity to retry the BeginSend. This can be implemented by sequentially calling back all pending sends.

    Besides, the gain of using async sends with TCP is very small if no thread safety guarantee stands (why make async sends in the general case if I can only have one concurrent writer and have to waste thread time synchronizing?).

    Can someone at Microsoft's networking team with authoritative information on this matter please share it in order to end this debate for good?

    By the way, are thread safety guarantees different depending on the use of the IAsyncResult pattern or the SocketAsyncEventArgs pattern?

    Thank you very much.





    • Edited by mulhacén Thursday, February 28, 2013 7:48 PM
    • Moved by Mike Feng Monday, March 4, 2013 4:26 AM
    Thursday, February 28, 2013 7:34 PM

All replies

  • First, TCP requires a connection.  What you posted refers to a connectionless protocol.

    Second, BeginSend makes a connection and has nothing to do with the actual data transmission.  It is the event that gets called when the connection completes. EndSend occurs when the connection is closed.  A TCP conmnection will remain open forever even after all the data is transmitted.  One end of the connection has to start a close connection for the EndSend to occur.  A connection can also close in the Net Library if more than 5 retries occur.  TCP sends data in packets or approximately 1500 bytes max.  Each packet is acknowleged by the recieving end of the connection and if the ack is not received within specified period of time (usally 5 seconds) the packet will get sent again.

    There is not much difference between the Send and the ASYNC send and I normally just use Send.  I believe the Send doesn't wait for all the ACK to occur.  The send and async send just forwards the data down the execution stack to the ethernet driver.  The timer tick services does tthe actual moving of the data.

    TCP also supports KeepAlive (optional) where packets of zero data bytes are sent when no data is sent to verify the other end of the connection is still functioning.  A TCP connection will stay open even when the ethernet cable is disconnected from the computer unless keepalive is used.


    jdweng

    Thursday, February 28, 2013 7:57 PM
  • Hi Joel,

    I appreciate your effort in answering, but honestly what it proves to me is that you have little knowledge of how .NET's async socket patterns work.

    Your answer is of little help to me. My question pertains to a very specific behavior inside .NET and Winsock.

    I recommend you to read MSDN's documentation pages on BeginSend and EndSend to get a grasp of the pattern.

    Regards.

    Thursday, February 28, 2013 8:41 PM
  • The BeginSend operations will occur in the order in which they are queued, but a subsequent BeginSend operation cannot begin until EndSend is called on the previous operation.  In your example you are pretty much guaranteed that the second "concurrent" call to BeginSend will execute before the callback of the first BeginSend can make a second call to send the rest of the data.

    But this is a very odd scenario... why would you begin a second write operation to the same endpoint without verifying the result of the first write operation?  I'm not sure that you can call this a problem with the Async pattern in .Net when the scenario seems unrealistic...

    The purpose of the Async send is that your code doesn't have to wait for the send to complete; it can go on to do other things while the data is sending - and in many cases this is much easier than doing your own multithreading.  I'm not sure that what you describe is in-line with the intended usage.

    Here's a simple example that shows the order of the callbacks with back-to-back calls, or with a delay between them:

    Public Class Form1
        Dim msg1() As Byte = System.Text.Encoding.ASCII.GetBytes("Bob")
        Dim msg2() As Byte = System.Text.Encoding.ASCII.GetBytes("Mary")
        Dim msg3() As Byte = System.Text.Encoding.ASCII.GetBytes("Fred")
    
        Dim callbackCount As Integer = 0
    
        Private Sub Form1_Load(sender As System.Object, e As System.EventArgs) Handles MyBase.Load
            Dim ms As New System.IO.MemoryStream
            ms.BeginWrite(msg1, 0, msg1.Length, AddressOf WriteCallback, ms)
            'add the delay below to get the messages in order
            'System.Threading.Thread.Sleep(100)
            ms.BeginWrite(msg3, 0, msg3.Length, AddressOf WriteCallback, ms)
        End Sub
    
        Private Sub WriteCallback(result As IAsyncResult)
            callbackCount += 1
            Dim ms As System.IO.MemoryStream = CType(result.AsyncState, System.IO.MemoryStream)
            ms.EndWrite(result)
            If callbackCount = 1 Then
                ms.BeginWrite(msg2, 0, msg2.Length, AddressOf WriteCallback, ms)
            ElseIf callbackCount = 3 Then
                MessageBox.Show(System.Text.Encoding.ASCII.GetString(ms.ToArray))
            End If
        End Sub
    End Class
     


    Reed Kimble - "When you do things right, people won't be sure you've done anything at all"

    Thursday, February 28, 2013 9:54 PM
    Moderator
  • Maybe I said too much.  The documentation you are reading does not apply to TCP.

    TCP requires a connection.  What you posted refers to a connectionless protocol.


    jdweng


    Friday, March 1, 2013 5:22 AM
  • Hi Reed,

    My distaste of the async pattern if BeginSend cannot be called concurrently comes from the availability of better choices to do the work and return as soon as possible. For example, a ConcurrentQueue (some kind of queueing is needed anyway to build the single threaded consumer). And it is hard to write a well behaving single thread pattern with the managed ThreadPool, so an explicit thread seems to be a better choice for the consumer.

    Again, the described scenario is a nightmare and my conclusion is that it can't be true.

    As a next step, I have tested it.

    I have setup a TCP connection between two computers.

    In the client I have subsequently called BeginSend twice. The first time with a 1 GB buffer. The second time with a 1 byte buffer. The server consumes with a 1 kB buffer.

    The callback for the 1 GB operation is called immediately (even if we know it will take much longer) and EndSend reports 1 GB sent.

    Some time later, the callback for the 1 byte operation is called.

    So, even in a scenario specifically aimed at producing partial sends, they don't happen.

    I started asking for an authoritative response from Microsoft. Now I beg for it. The documentation about BeginSend, EndSend, SendAsync, etc... seems to be incomplete, suggesting in the general case that partial sends are to be expected, but they seem to never happen. Queueing sends in the application and writing a single threaded sender seems to be unnecessary and inefficient. Am I right?

    Thank you very much.





    • Edited by mulhacén Tuesday, March 5, 2013 2:42 PM
    Tuesday, March 5, 2013 12:51 PM
  • Hi Mulhacen,

    came across this thread searching for the solution if the exact same problem, did you finally find the required information? Since there is an obvious conflict in the BeginSend and EndSend MSDN documentation, it is really not clear if a BeginSend queues all data in an atomic way - in respect to other (Begin)Sends. My idea is (not tested until now) that it can be different for blocking and nonblocking sockets. So I suspect the default behaviour is: BeginSends queues all data (or there will an exception) for blocking sockets (the default) and BeginSend might not (i.e. EndSend may succeed with less bytes sent) queue all bytes using a nonblocking socket. I plan to test this assumption, but maybe you already know the solution, the thread is a bit old.

    Note to jdweng: Please do not reply to anything you do not understand. Thanks.

    Alex

    Monday, September 30, 2013 4:56 PM
  • I'm not going to answer your question, but help you with the solution.  There is an ethernet driver on the PC that may effect the way the queing is handled.  The Net library passes packets to the ethernet driver.  Using the timers tick process for passing data.  TCP packets are sent by the ethernet driver at max size of around 1500 bytes according to the RFC specifcation.  Each packet get an acknowledgement back from the recieving end that gets handled by the ethernet driver.  I don't think this ack gets sent back to the Net library.

    I believe a stream to connect the ethernet driver with socket in the Net library.  If this is the case, the timer tick process will empty the transmit buffer approximately every 10msec and send data to the ethernet driver.

    I also think the EndSend occurs when the timer tick process finishes removing all the data from the tx stream.  Not that the data is actually sent by the ethernet driver.  I don't think there is going to be much difference between the blocking and non blocking functions for 2 reasons

    1) The Net library doesn't get confirmation of the TCP ACKs

    2) The Net library is only checking the TX stream  between the net library and the ethernet driver.  Not the buffer in the ethernet driver.


    jdweng

    Monday, September 30, 2013 5:13 PM
  • FYI, I have opened a connect issue on this in the hope to finally find the solution.

    https://connect.microsoft.com/VisualStudio/feedback/details/804029/socket-beginsend-and-socket-endsend-documentation-conflict

    Alex

    Tuesday, October 1, 2013 8:38 AM