none
Using Streams reliably, Stream.Write() method doesn't indicate number of bytes written RRS feed

  • Question

  • Hello,

    I'm trying to understand the programming model of the System.IO.Stream abstract class, particularly the Write() function and why it doesn't return information about the (partial) success of a write.

    The basic functions of Windows supports knowing how many bytes were written to a "stream". For example the system WriteFile(hFile, lpBuffer, nNumberOfBytesToWrite, lpNumberOfBytesWritten, lpOverlapped.

    So when i implement my own buffering I know what has been sent and what has not. This is especially useful for very slow connections. I can then update my buffer dependent on what was actually written.

    But when I look at the Stream.Write() interface, it doesn't provide this information, instead raising an exception (with no information about how much was written) or assumes that all data was written.

    Worse, some implementations (like System.IO.Ports.SerialPort.SerialStream,4.0.0.0) completely ignore the number of bytes written and provides success if one or more bytes were written. Thus, if I request to write 5000 bytes, only 4000 were written, that would be treated as success and the remaining 1000 bytes are effectively lost.

    Can someone link me to resources that describe how to handle the above situation, or provide reasoning behind the design of the API, with possible solutions?

    Thanks,
    Jason.


    • Edited by Jason Curl Sunday, August 5, 2012 7:28 AM
    • Moved by Mike FengModerator Friday, August 10, 2012 1:30 AM BCL (From:Common Language Runtime)
    Sunday, August 5, 2012 7:27 AM

Answers

  • Hi Jason,

    When you make a new class inherits the base Stream class, you add one more overload function. 

    >>What happens if flow control allows that I can write half the data, but flow control is active for the rest?

    You have control this flow, you can close it or keep it, it is just designed by yourself.

    Best regards,


    Mike Feng
    MSDN Community Support | Feedback to us
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    • Marked as answer by Jason Curl Thursday, August 23, 2012 8:00 AM
    Wednesday, August 22, 2012 2:35 PM
    Moderator

All replies

  • "Worse, some implementations (like System.IO.Ports.SerialPort.SerialStream,4.0.0.0) completely ignore the number of bytes written and provides success if one or more bytes were written. Thus, if I request to write 5000 bytes, only 4000 were written, that would be treated as success and the remaining 1000 bytes are effectively lost."

    You can get the NumberOfBytesWritten through reflection of numBytes in the Write method.  If you're serial communication is critical, don't use the .NET SerialPort.  This is just one of its many problems.

    Sunday, August 5, 2012 10:45 AM
  • Thanks for the reply. I'm working on implementing my own Serial stream although this is half of the story. The goal is to have a stream that talks to remote terminals, be it serial, telnet or ssh for embedded development (I have all three in my box). Other protocols can be tacked on later if required.

    So I'm looking on how should streams handle the case that a Write() doesn't possibly write all data?

    Is it expected that if a write works it writes data but if it doesn't work then no data is written at all? It's not at all clear, the only vague statement from System.IO.Stream is "If an exception occurs, the position within the stream remains unchanged." And I'm not even sure how this is possible, given that some exceptions aren't even known until one attempts to write.

    Sunday, August 5, 2012 1:13 PM
  • Hi Jason,

    Thank you for posting on this forum.

    Which issue do you want to focus on?

    If you focus on Why Stream.Write method doesn't return the success bytes numbers, I would like to move this thread to BCL forum: http://social.msdn.microsoft.com/Forums/en-us/netfxbcl/threads 

    If you focus on Why serialPort.Write method doesn't return the success bytes numbers it sent, I would like to move this thread to Network forum: http://social.msdn.microsoft.com/forums/en-US/ncl/threads/ 

    In addition, so far, this feature doesn't exist, if it is very important to you, please submit it here: http://connect.microsoft.com/VisualStudio/ Thank you.

    Best regards,


    Mike Feng
    MSDN Community Support | Feedback to us
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Thursday, August 9, 2012 10:08 AM
    Moderator
  • Hello Mike,

    Yes, the issue is that the Write() method doesn't return the number of bytes written, but more importantly, how should I implement my own streams for reliable behaviour?

    So, moving to the BCL forum might be the best way.

    Regards,
    Jason.

    Thursday, August 9, 2012 5:31 PM
  • Hi Jason,

    I have moved this thread.

    Back to this issue: You can try the API you have mentioned: http://msdn.microsoft.com/en-us/library/windows/desktop/aa365747(v=vs.85).aspx 

    And here is a way to get the stream handle for the first parameter in this API: http://stackoverflow.com/questions/8640660/how-to-get-the-underlying-file-handle-of-a-stream 

    I hope this will be helpful.

    Best regards,


    Mike Feng
    MSDN Community Support | Feedback to us
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Friday, August 10, 2012 1:43 AM
    Moderator
  • Thanks Mike.

    Let's say I'm using WriteFile() as my base implementation. But I'm now writing a .NET stream component that other software should use. What is the programming model for Stream.Write() that should map to WriteFile()? I don't want to lose data, instead buffering everything and assuming that there is flow control of some sorts.

    A WriteFile() may return lpNumberOfBytesWritten which is not equal to nNumberOfBytesToWrite. How should the stream.Write() behave in this case?

    If I create my own buffered stream class, something similar may also occur: a Write() for 100 bytes could occur, but the buffer has only 50 bytes free (and then I use the WriteFile() and know how much from my buffer was consumed and can free, allowing unwritten data to be retried at a later time). Is there a mechanism via the Stream class to provide partial write success and to tell the user how much write was successful? I couldn't find such a mechanism.

    My assumptions lean towards the scenario that I should only accept a Write() if the data can be completely buffered, and if not, then raise an exception without buffering any data. Is this the expected programming model for Streams?

    Friday, August 10, 2012 9:09 AM
  • Hi Jason,

    >>A WriteFile() may return lpNumberOfBytesWritten which is not equal to nNumberOfBytesToWrite. How should the stream.Write() behave in this case?

    Does not returning lpNumberOfBytesWritten the your expect?

    >>Is this the expected programming model for Streams?

    It sounds a good idea, you can try it.

    Best regards,


    Mike Feng
    MSDN Community Support | Feedback to us
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Monday, August 13, 2012 6:06 AM
    Moderator
  • Hello Mike,

    Sorry if it sounds we're going back and forth, I'm probably not making something clear. I will be using WriteFile() in my implementation. But the Stream classes don't offer as a return the value lpNumberOfBytes, instead they have a return signature type of "void".

    So what should a user of .NET's stream class expect if the underlying operating system can't write all of the data for whatever reason? This interface/use case isn't well documented in MSDN.

    Take a stream with flow control (e.g. GPIB, Serial, TCP). What happens if flow control allows that I can write half the data, but flow control is active for the rest? I fall into a Timeout case where not all data was written, or I say success with only half the data written, how does a .NET Stream class communicate how much data was written? If it's either all or none, or data is lost, that's fine, I'm looking for an authoritative answer which is the expectation also from 95% all other .NET users.

    Thanks in advance,
    Jason.

    Monday, August 13, 2012 5:27 PM
  • Hi Jason,

    When you make a new class inherits the base Stream class, you add one more overload function. 

    >>What happens if flow control allows that I can write half the data, but flow control is active for the rest?

    You have control this flow, you can close it or keep it, it is just designed by yourself.

    Best regards,


    Mike Feng
    MSDN Community Support | Feedback to us
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    • Marked as answer by Jason Curl Thursday, August 23, 2012 8:00 AM
    Wednesday, August 22, 2012 2:35 PM
    Moderator
  • Thanks Mike. So the answer I'm looking for is "it's implementation defined". Likely then the only thing a stream could do in this case is to raise an exception and throw away the data and try to have all the handling in the base stream class itself if possible (as it's closer to the Win32 API).
    Thursday, August 23, 2012 7:59 AM