locked
Endian swapping with doubles and floats RRS feed

  • Question

  • I've been working on a game using XNA which uses custom made binary files. These files are being created (serialised) out from a tool I wrote. Now I'm porting it to the 360 I'm having to perform endian swapping on any variable instances I deserialise from the binaries. Luckily enough, most of the variables I'm serialising out to the file are uints so I just wrote this simple little function to swap the bytes after I deserialise them:

            public static void EndianSwap(ref uint var) 
            { 
                var = ( 
                    (0x000000ff & (var >> 24)) |  
                    (0x0000ff00 & (var >> 8)) | 
                    (0x00ff0000 & (var << 8)) | 
                    (0xff000000 & (var << 24)) 
                    ); 
            } 

    The problem I've hit is that I'm also serialising out doubles too. It's easy enough to extend the above function to 8 bytes, which would work fine if the doubles were unsigned (I know this is inherently impossible), but I'm not sure how C# handles the signage. I'm assuming it uses the highest bit as a sign bit but I'm not sure how I'd extend the above function to handle this.

    Any ideas?

    Sunday, September 7, 2008 9:41 AM

Answers

  • Just for completeness, protocol buffers *is* a binary format - it is the binary format Google invented (and use on vast ranges of their products).

    Re hacking levels, well, ultimately the code can be reversed. You'd have to use signing / hashing (of the level data) to prevent that...

    Re reversing the byte[] - you need to be careful to only reverse each block of bytes, which can get tricky: you might need to understand everything *before* you deserialize it just to do the fixup, which is a pain.

    Note that for MiscUtil you won't want the entire dll - just the one file (or maybe a couple) - but how suitable it is depends on your exact use-case.

    Marc
    • Edited by Marc GravellMVP Sunday, September 7, 2008 7:13 PM clarified haqshing of level data
    • Proposed as answer by jack 321 Wednesday, September 10, 2008 7:18 AM
    • Marked as answer by jack 321 Thursday, September 11, 2008 2:46 AM
    Sunday, September 7, 2008 7:12 PM

All replies

  • Personally I would fix endianness while it is in byte[] form - i.e. before it has a type. In particular, for some types you could get a runtime exception / irretrievable data corruption simply by trying it the wrong way.

    For reference, Jon Skeet has an a sample EndianBitConverter in his "MiscUtil" library that might be handy, or just do it manually by reversing portions of the array.


    As another alternative, consider a serialization library that handles endianness for you - for example, "protocol buffers" would allow simpel (but very efficient, both CPU and bandwidth) serialization of typical objects; a .NET implementation of this is here:


    (I haven't tested it with XNA, but it works find on .NET / CF / mono / Silverlight; I'd be happy to investigate and fix any specific issues you get... in *theory* the endianness works, but I haven't tested it on a 360)

    Marc
    Sunday, September 7, 2008 9:48 AM
  • I've just had a quick look at protobuf-net and it looks like a very cool serialisation library. Thing is, I'm wanting to use binary serialisation as I don't want a format that is easily editable by the user. Don't want the levels and settings of my games to be hacked to buggery by everyone :)

    The CF doesn't actually support any formatters so I'm using CompactFormatter which works fine on both 360 and PC.

    Good suggestion about swapping the endianness while it's in byte[] form. CompactFormatter only serialises and deserialises objects but as long as the objects that I give it are bytes it should work fine. It just means I'll have to convert to and from byte[] before serialisation and after deserialisation respectively.

    I'll have a look at the MiscUtil library too as there's no point in re-inventing the wheel if it does what I want it to do.

    Sunday, September 7, 2008 10:43 AM
  • Just for completeness, protocol buffers *is* a binary format - it is the binary format Google invented (and use on vast ranges of their products).

    Re hacking levels, well, ultimately the code can be reversed. You'd have to use signing / hashing (of the level data) to prevent that...

    Re reversing the byte[] - you need to be careful to only reverse each block of bytes, which can get tricky: you might need to understand everything *before* you deserialize it just to do the fixup, which is a pain.

    Note that for MiscUtil you won't want the entire dll - just the one file (or maybe a couple) - but how suitable it is depends on your exact use-case.

    Marc
    • Edited by Marc GravellMVP Sunday, September 7, 2008 7:13 PM clarified haqshing of level data
    • Proposed as answer by jack 321 Wednesday, September 10, 2008 7:18 AM
    • Marked as answer by jack 321 Thursday, September 11, 2008 2:46 AM
    Sunday, September 7, 2008 7:12 PM