none
Difference between CE_OVERRUN and CE_RXOVER in COMSTAT errors RRS feed

  • Question

  • Hi!
    I have some code that reads data coming into the serial port of my computer running Windows XP. I need to detect a buffer overflow when it happens at the port. As far as I can see I need to check the error coming back from ClearCommError(). What I do not know is which one should I check for; CE_OVERRUN or CE_RXOVER ?
    Any help will be much appreciated.
    Thanks
    Ruchira
    Thursday, October 15, 2009 1:46 PM

Answers

  • Hmya, to be tediously argumentative, this isn't quite correct.  There are two buffers involved.  One is on the chip that you plug the cable into.  It buffers received bytes and gives the device driver some latitude in responding to the interrupt.  This buffer used to be one byte deep back in the stone age but is now 16 or 256 bytes.  If this buffer overflows, you get CE_OVERRUN.  If this happens, there's something seriously messed up with one of the device drivers in the operating system.

    The second buffer lives in the device driver, it gives your program some latitude in reading the bytes.  It is sizable with the dwInQueue argument of SetupComm().  You should increase the default size when you expect your program to be slow to read.  If you don't read fast enough, you'll get CE_RXOVER.  That has much less to do with the size of the buffer you pass to ReadFile(), much more with calling ReadFile() frequently enough. 

    The primary way to avoid that mishap is by enabling handshaking, that tells the device to stop sending when the buffer is filling up.  The best way to avoid it is to make your program responsive to events signaled by the driver.  You'd use WaitCommEvent().  That kills two birds with one stone, you'll get error notifications as well.

    Dealing with these kind of errors isn't actually very useful.  It is kinda like your program crashing with an access violation.  Nice to know, but the program is still dead.  The larger source of problems with serial port communication is electrical noise.  RS-232 isn't a very good signaling standard, it only allows a 10 feet cable at 9600 baud.  Something that everybody always ignores.  Electrical noise makes bits flip state randomly.  You can get some protection against that by using parity.  That isn't very reliable though, it cannot detect two flips. 

    Reliable serial communication always requires a protocol, an independent way to verify that the data that was sent matches the data that was received.  Error checking is done with CRCs, just like it is done in TCP/IP.  Except that you have to do it yourself, you'll get no help from the hardware.   With that in place, you really don't care about the serial port errors anymore, they are covered by the much more reliable error detection provided by the protocol.  Plus the protocol now actually helps you recover from a data error.  Note that this argument is mute if this is actually a USB device whose driver emulates a serial port.

    Hans Passant.
    • Marked as answer by RuchiraM Friday, October 16, 2009 7:04 PM
    Thursday, October 15, 2009 6:16 PM
    Moderator

All replies

  • CE_OVERRUN is a hardware or driver error servicing the port, CE_RXOVER is an error due to your program being too slow or the buffer you supply being too small.  Reference:

    http://msdn.microsoft.com/en-us/library/ms810467.aspx
    Thursday, October 15, 2009 2:18 PM
  • Hi! Scott,
    Thanks for your reply. So I need to check for CE_RXOVER. You mention that CE_RXOVER is an error due to my program being too slow or the buffer I supply being too small. I don't see (in code)  where I am supplying a buffer to ClearCommError.

    thanks
    Ruchira
    Thursday, October 15, 2009 2:22 PM
  • It's about the buffer you supplied to ReadFile.
    Thursday, October 15, 2009 3:49 PM
  • Hi! Scott,
    Please see the code below. ClearCommError() sets the dwErrorFlags (this tells me whether dwErrorFlags returns CE_RXOVER). However the call to ReadFile is made much later in the code and the buffer that is input to ReadFile is what I pass into the function ReadData. So how is the buffer overflow referrign to the buffer I pass as input to ReadFile ? What am I missing here.
    Thanks in advance,
    Ruchira

    int

     

    Serial::ReadData(void *buffer, int limit)
    {
        if(!m_bOpened || m_hIDComDev == NULL) return(0);
        BOOL bReadStatus;
        DWORD dwBytesRead, dwErrorFlags;
        COMSTAT ComStat;

     

        ClearCommError(m_hIDComDev, &dwErrorFlags, &ComStat);

        if(!ComStat.cbInQue)
            
    return(0);
        if(dwErrorFlags == CE_RXOVER)
       { 
           m_bBufferOverflow =
    true;
        }
        dwBytesRead = (DWORD) ComStat.cbInQue;

     

        if(limit < (int) dwBytesRead) dwBytesRead = (DWORD)limit;
        bReadStatus = ReadFile(m_hIDComDev, buffer, dwBytesRead, &dwBytesRead, &m_OverlappedRead);

     

       if( !bReadStatus )
       {

          if(GetLastError() == ERROR_IO_PENDING)
         {
            WaitForSingleObject( m_OverlappedRead.hEvent, 2000);
             return((int)dwBytesRead);
         }

     

       return(0);

      }

     

      return((int)dwBytesRead);

    }

    Thursday, October 15, 2009 4:02 PM
  • There aren't going to be any read errors before you call ReadFile (unless they are left over from a previous call to ReadFile). Why don't you check for errors after you perform the operation?
    Thursday, October 15, 2009 5:25 PM
  • Thanks Scott, I am doing that now. Also I need to check for buffer overflow at the hardware level. So I am using CE_OVERRUN ? Is this correct ?
    thanks in advance
    Ruchira
    Thursday, October 15, 2009 5:35 PM
  • Scott, one more question. How can one determine the size of the input buffer at the hardware level?
    Thursday, October 15, 2009 5:54 PM
  • Hmya, to be tediously argumentative, this isn't quite correct.  There are two buffers involved.  One is on the chip that you plug the cable into.  It buffers received bytes and gives the device driver some latitude in responding to the interrupt.  This buffer used to be one byte deep back in the stone age but is now 16 or 256 bytes.  If this buffer overflows, you get CE_OVERRUN.  If this happens, there's something seriously messed up with one of the device drivers in the operating system.

    The second buffer lives in the device driver, it gives your program some latitude in reading the bytes.  It is sizable with the dwInQueue argument of SetupComm().  You should increase the default size when you expect your program to be slow to read.  If you don't read fast enough, you'll get CE_RXOVER.  That has much less to do with the size of the buffer you pass to ReadFile(), much more with calling ReadFile() frequently enough. 

    The primary way to avoid that mishap is by enabling handshaking, that tells the device to stop sending when the buffer is filling up.  The best way to avoid it is to make your program responsive to events signaled by the driver.  You'd use WaitCommEvent().  That kills two birds with one stone, you'll get error notifications as well.

    Dealing with these kind of errors isn't actually very useful.  It is kinda like your program crashing with an access violation.  Nice to know, but the program is still dead.  The larger source of problems with serial port communication is electrical noise.  RS-232 isn't a very good signaling standard, it only allows a 10 feet cable at 9600 baud.  Something that everybody always ignores.  Electrical noise makes bits flip state randomly.  You can get some protection against that by using parity.  That isn't very reliable though, it cannot detect two flips. 

    Reliable serial communication always requires a protocol, an independent way to verify that the data that was sent matches the data that was received.  Error checking is done with CRCs, just like it is done in TCP/IP.  Except that you have to do it yourself, you'll get no help from the hardware.   With that in place, you really don't care about the serial port errors anymore, they are covered by the much more reliable error detection provided by the protocol.  Plus the protocol now actually helps you recover from a data error.  Note that this argument is mute if this is actually a USB device whose driver emulates a serial port.

    Hans Passant.
    • Marked as answer by RuchiraM Friday, October 16, 2009 7:04 PM
    Thursday, October 15, 2009 6:16 PM
    Moderator
  • If that is tediously argumentative, then I'm in trouble. That is 100% correct in my experience. The CE_OVERRUN has (mostly) been dealt with by the UART manufacturers. (The latest [16750 clones] have 64 byte buffers internally, iirc, and the driver can set the size that triggers the ISR, but...I know of no way to change that programmatically now.)

    The comments wrt USB also echo similar observations you have made recently with respect to the inconsistent hubbing (ferrying) of RS-232 over USB, and the lack of real hardware handshaking. Its a shame when hardware engineers are cut out of software development.

    Framing errors (used to be) a good indication that you had a noisy line, and RS-422 line drivers have typically been used to eliminate the electrical noise in such communications systems. (Its twisted pair, so the noise cancels itself out.)

    When I had better control of the port, these things were "simple" to deal with. I could achieve sustained 115200bps on a 486DX/100 while lighting off Word in WFW3.11. That is about as good as it gets, I think. Protocols and error checking were very minimal, and the UART itself was within easy reach.

    Anyway, Hans...that is yet another very uselful posting about a very vexing problem. Thanks for the insight.
    Thursday, October 15, 2009 6:35 PM
  • Hans,
    Thank you so much for your reply and the information.
    regards,
    Ruchira
    Friday, October 16, 2009 7:04 PM