none
Converting char buffer to unsigned short/unsigned int - Big/Little Endian RRS feed

  • Question

  • I am dealing with DIS(Distributed Interactive Simulation ) protocol packets which i am receiving on the the port using UDP.

    These packets are formatted in Big-Endian. Now i want to receive the packets on the Windows machine in my application.

    For UDP socket programming i am binding socket with the following parameters.

     
    struct sockaddr_in RecvAddress={0}; 
     
    RecvAddress.sin_family = AF_INET
    RecvAddress.sin_port = htons( 3000 ); 
    RecvAddress.sin_addr.s_addr = htonl(INADDR_ANY); 
     

    So once i receive the data in my char * buffer i need to check the header values.
    The header information is in Big-Endian format.
    The header is given below
     
    typedef struct 
        unsigned char    ProtocolVersion ; 
        unsigned char    ExerciseID ; 
        unsigned char    PDUType ; 
        unsigned char    ProtocolFamily ; 
        unsigned long    TimeStamp ; 
        unsigned short   Length ; 
        unsigned short   Padding ; 
    }PDUHeader; 
     

    If the data received was in Little Endian format, i would have used memcpy() directly and populated the data of the header directly. But the data recieved is in Big-Endian format.

    Question 1:- Is there any way to directly obtain the data in my buffer in Little endian format and directly use to memcpy() to speeden up the process. I was going through the socket APIs and tried ntohl() function in setting the sockaddr_in RecvAddress parameters thinking it may serve the purpose. But somehow it didnt workout.

    Since i currently couldnt find a way, i took the existing Big-endian buffer and started extracting information byte by byte.

    So i follow the process below.
    I simply use the values from Buffer for the first four variables of header as they are of char type.
    But for the long and short i made wrote functions which swapped the buffer and then converted them into short/long as per requirement.

    For unsigned short i wrote this function below, so far its working for me. I would like inputs if this is correct approach or if any better method is available.

    short bytes2ShortK(char *szBuffer) 
        static  unsigned char arrShort[2]=""; 
        ZeroMemory(arrShort,sizeof(arrShort)); 
        // extract the two bytes to be converted to short from the complete PDU 
        arrShort[1] = szBuffer[0]; 
        arrShort[0] = szBuffer[1]; 
     
        /* As short consists of 2 bytes, the MSB is shifted 8 bits left and then  
        is OR'd with the value of LSB to obtain short  */ 
        return (arrShort[1]<<8)|(arrShort[0]); 
     


    I want to have a similar code for the char * buffer to unsigned as well as signed integer. Any inputs would be appreciated. I want to know if my code above is correct for unsigned short or will work only for signed short.


    Kavitesh Singh.
    Thursday, July 17, 2008 12:04 PM

Answers

  • I found the solution with some hit and trial.

        memcpy(&PduHeader,buffer,sizeof(PDUHeader)); 
         
        PduHeader.TimeStamp = ntohl(PduHeader.TimeStamp); 
        PduHeader.Length = ntohs(PduHeader.Length); 
        PduHeader.Padding = ntohl(PduHeader.Padding); 
     

    I had done the ntohl in the address but i had to do it for the variable also to get converted back to little endian.

    Kavitesh Singh.
    Monday, July 21, 2008 6:30 AM

All replies

  • Oh one more thing.

    The header information is in Big-Endian but the actual message is in Little-Endian format.

    So i need to have initial header only converted to little-endian (if that is possible) and keep the data part as it is.

    Kavitesh Singh.
    Thursday, July 17, 2008 12:34 PM
  • There are CRT functions you could use: _swab(), _byteswap_ulong() and _byteswap_ushort().
    Hans Passant.
    Thursday, July 17, 2008 7:13 PM
    Moderator
  • The swab function works only in pairs which can be a problem considering the above struct where all the elements are not in pairs.

    Moreover any tips of converting char* to usigned/signed int would be appreciated. The code above for converting char* to  unsigned short is correct?????

    Kavitesh Singh.
    Friday, July 18, 2008 4:18 AM
  • Kavitesh Singh said:

    The swab function works only in pairs which can be a problem considering the above struct where all the elements are not in pairs.


    That sentence makes no sense. ASCII strings and chars are the same in little-endian and big-endian. Only shorts, ints, longs, long longs and I believe floats and doubles need to be flipped. Using the CRT functions pointed out by nobugz should be completely sufficient. Don't invent your own.  

    (I hasten to add I'm fuzzy on whether floating points need to be lumped in with these, but I suspect so).
    • Edited by Brian Muth Friday, July 18, 2008 4:31 AM add caveat
    Friday, July 18, 2008 4:29 AM
  • Brian Muth said:

    That sentence makes no sense. ASCII strings and chars are the same in little-endian and big-endian. Only shorts, ints, longs, long longs and I believe floats and doubles need to be flipped. Using the CRT functions pointed out by nobugz should be completely sufficient. Don't invent your own.  


    I use the code given on MSDN for the swab() function

    #include <stdlib.h> 
    #include <stdio.h> 
     
    char from[20] = "12ab34cd56ef"; 
    char to[20] =   ""; 
     
    int main() 
        printf( "Before: %s\n", from ); 
        _swab( from, to, sizeof( from ) ); 
        printf( "After:  %s\n",to ); 
     
    Output: 
    Before: 12ab34cd56ef 
    After:  21ba43dc65fe 


    So if you see clearly the bytes are swapped in pair. This is even number of bytes which was in the char buffer.

    Now i change it and have a look at the output.

    #include <stdlib.h> 
    #include <stdio.h> 
     
    char from[20] = "12ab34cd56efX"; 
    char to[20] =   ""; 
     
    int main() 
        printf( "Before: %s\n", from ); 
        _swab( from, to, sizeof( from ) ); 
        printf( "After:  %s\n",to ); 
     
    Output: 
    Before: 12ab34cd56efX 
    After:  21ba43dc65fe 

    You would observe, the X has gone missing when i deal with the odd number of bytes.

    I hope its all clear to you about the issue i was talking.

    Now if i take the example of my struct which was in Big-Endian format. It will goof up the byte locations if i directly use the swab() function.

    To avoid this issue, i take the data from 4(starting index - 0) to 7 for the long and then swap it manually(no inbuilt functions) and try to read it as the long value.

    Similarly for short which would be in 8 and 9 bytes, i use my program above. I need inputs for the function which can convert unsigned int from char buffer value.

    I would have put it this way, the question.
    Is there any way to convert the byte buffer into unsigned/signed short and unsigned/signed int?








    Kavitesh Singh.
    Friday, July 18, 2008 6:20 AM
  • It's not possible to convert the buffer, you have to convert each individual struct member.  With one long and 2 shorts, that's 3 lines of code. 
    Hans Passant.
    Friday, July 18, 2008 9:41 AM
    Moderator
  • nobugz said:

    It's not possible to convert the buffer, you have to convert each individual struct member.  With one long and 2 shorts, that's 3 lines of code. 


    Hans Passant.


    I agree that at one go i cant do the conversion straight away.

    For this once i have the buffer, i take 2 bytes or 4 bytes depending on the short/int.
    I then use bytes2ShortK() defined above for short and get the result.
    I was looking for something which would deal with int in similar way.

    i tried the following but it doesnt give me correct values.

    unsigned int bytes2UIntK(char *szBuffer) 
        static unsigned char arrUInt[4]=""; 
        static unsigned int intBits = 0
        // extract the four bytes to be converted to integer from the complete PDU 
        ZeroMemory(arrUInt,sizeof(arrUInt)); 
        intBits = 0
     
        arrUInt[0] = szBuffer[3]; 
        arrUInt[1] = szBuffer[2]; 
        arrUInt[2] = szBuffer[1]; 
        arrUInt[3] = szBuffer[0]; 
     
        intBitsintBits = intBits|(arrUInt[0]<<24); 
        intBitsintBits = intBits|(arrUInt[1]<<16); 
        intBitsintBits = intBits|(arrUInt[2]<<8); 
        intBitsintBits = intBits|(arrUInt[3]); 
     
        return intBits; 


    Kavitesh Singh.
    Friday, July 18, 2008 12:56 PM
  • You are swapping twice.  As Brian recommended, it's best to avoid re-inventing CRT functions.
    Hans Passant.
    Friday, July 18, 2008 2:54 PM
    Moderator
  • I would imagine that this line of code will fail on some compilers:

        return (arrShort[1]<<8)|(arrShort[0]);

    If you don't promote arrShort to short prior to the left shift, I would imagine you will always get zero for the high order byte.  Try:

        return ((short)arrShort[1]<<8)|(arrShort[0]);

    Friday, July 18, 2008 2:57 PM
  •  Kavitesh, I don't see the light bulb flickering over your head yet. You are overly focused on accepting a character string as your input.

    Consider only using two functions: _byteswap_ushort() and _byteswap_ulong().

    Example:

    PDUHeader x = getMyBigEndianPDUHeader();

    x.TimeStamp = _byteswap_ulong(x.TimeStamp);
    x.Length = _byteswap_ushort (x.Length);
    x.Padding = _byteswap_ushort(x.Padding);

    // done.




    • Edited by Brian Muth Friday, July 18, 2008 5:40 PM spelling
    Friday, July 18, 2008 5:13 PM
  • Brian Muth said:

     Kavitesh, I don't see the light bulb flickering over your head yet. You are overly focused on accepting a character string as your input.

    Consider only using two functions: _byteswap_ushort() and _byteswap_ulong().

    Example:

    PDUHeader x = getMyBigEndianPDUHeader();

    x.TimeStamp = _byteswap_ulong(x.TimeStamp);
    x.Length = _byteswap_ushort (x.Length);
    x.Padding = _byteswap_ushort(x.Padding);

    // done.




    i tried the following code on your advice:

    memcpy(&PduHeader,buffer,sizeof(PDUHeader)); 
    PduHeader.TimeStamp = _byteswap_ulong(PDUHeader.TimeStamp); 
    PduHeader.Length = _byteswap_ushort(PDUHeader.Length); 
    PduHeader.Length = _byteswap_ushort(PDUHeader.Length); 
     

    The values DONT turn up as expected. Any other way to do it??


    Kavitesh Singh.
    Monday, July 21, 2008 5:32 AM
  • I found the solution with some hit and trial.

        memcpy(&PduHeader,buffer,sizeof(PDUHeader)); 
         
        PduHeader.TimeStamp = ntohl(PduHeader.TimeStamp); 
        PduHeader.Length = ntohs(PduHeader.Length); 
        PduHeader.Padding = ntohl(PduHeader.Padding); 
     

    I had done the ntohl in the address but i had to do it for the variable also to get converted back to little endian.

    Kavitesh Singh.
    Monday, July 21, 2008 6:30 AM