locked
UWP and Net Standard data compression RRS feed

  • Question

  • Created some serialized, compressed resources in .NET Standard 2.0 using DeflateStream and GZipStream.

    I then embedded these resources in a UWP app.

    Using the "same" compression classes in UWP, I'm getting file sizes that differ from the original when I decompress.

    The output needs to be deserialized but of course won't because the file sizes are off.

    I can create a standard library that does both, but I would have thought that these classes should work the same in UWP and Standard when dealing with just Streams.

    Is this a bug or "by design"?


    GHS

    Thursday, October 10, 2019 2:54 AM

Answers

  • Hi,

    I might have misunderstood something. I tried again to compress the object in the .Net standard library and decompress it in the UWP project. And it works fine. I also checked with the size of MemoryStream. It's also the same.

    Here is the repo that I have made from your codes: CompressionSample. You can take a try.

    About decompressedStream.Length:

    DeflateStream.Length is not a supported attribute, you can F12 to see below info:

            // Summary:
            //     This property is not supported and always throws a System.NotSupportedException.
            //
            // Returns:
            //     A long value.
            //
            // Exceptions:
            //   T:System.NotSupportedException:
            //     This property is not supported on this stream.
            public override long Length { get; }

    Thanks,

    Elvis Xia



    Thursday, October 17, 2019 7:27 AM

All replies

  • Hi,

    Do you mean the DeflateStream and GZipStream class works well in .NET Standard 2.0, but not in UWP? When I use the DeflateStream and GZipStream in UWP, the decompressed file is the same size as the original file. So can you please provide a simple sample that can reproduce this issue and what kind of file you try to decompress?

    Best Regards,

    Fay


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Thursday, October 10, 2019 7:27 AM
  • Hi Fay,

    I meant that they appear to work, but the output from a NET Standard "compression" is not "decompressed" in UWP to the original size using the "same" classes. There is no error decompressing, but the file size is "smaller".

    (I first tried GZipStream, then DeflateStream; neither behaved as expected while the compressed file sizes were different).

    The source to the compression are binary serialized C# data "objects" that get deserialized at run time. The serialization and deserialization work fine (from Standard to UWP); it's the decompression from Standard to UWP that's an issue (when that step is added).

    The resource stream retrieved from the embedded resource in UWP is the same size as was added to UWP, so that step isn't a problem either.

    At this time, it's not practical to use "my data" as an example, but I'll see if I can reproduce it using something else. If the serialization output is what's causing the compression to be "unpredictable" then I may not be able to reproduce it with "something else".

    Short story, everything works, except I cannot seem to decompress "properly" in UWP what is compressed using NET Standard.

    Regards,

    Gerry



    GHS



    • Edited by bs777 Thursday, October 10, 2019 12:49 PM
    Thursday, October 10, 2019 12:42 PM
  • You really should look for an error in your code, not in the library code. Compression is actually not even part of the UWP, platform uses third-party library. Most likely, this is something simple, for example, you are closing these streams in the wrong order (consuming stream must be disposed before source stream to flush buffers).

    MCSD App Builder, MCSA UWP

    Thursday, October 10, 2019 4:56 PM
  • Thanks for the useless non-answer.

    GHS

    Thursday, October 10, 2019 6:31 PM
  • Hi Fay,

    I've reproduced (sort of) the problem. I used a small "object graph" as opposed to my "large" object graph, and in this case the stream compresses to "zero"; and that was simply limiting myself to a console app as opposed to UWP versus NET Standard.

    (With a larger graph I may get more output but can't see the point at this time).

    using System;
    using System.IO;
    using System.IO.Compression;
    using System.Runtime.Serialization.Formatters.Binary;

    namespace TestConsole {

       [Serializable]
       public class Person {

          public string FirstName { get; set; }
          public string LastName { get; set; }
       }

       /// <summary>
       ///
       /// </summary>
       class Program {

          /// <summary>
          ///
          /// </summary>
          static void Main( string[] args ) {

             Person p = new Person() { FirstName = "Bill", LastName = "Smith" };
             Run( p );
             Console.ReadKey();
          }

          /// <summary>
          ///
          /// </summary>
          public static void Run( Person p ) {

             // SERIALIZE.
             using ( MemoryStream originalStream = Serialize( p ) ) {

                Console.WriteLine( $"Original length: {originalStream.Length}" );   // 198.

                // COMPRESS.
                using ( MemoryStream compressedStream = new MemoryStream() ) {

                   DeflateStream compressionStream = new DeflateStream( compressedStream, CompressionMode.Compress );

                   originalStream.Position = 0;
                   originalStream.CopyTo( compressionStream );

                   Console.WriteLine( $"Compressed length: {compressedStream.Length}" );  // 0!

                   // VERIFY (closes compressed stream).
                   MemoryStream decompressedStream = Decompress( compressedStream );

                   Console.WriteLine( $"Decompressed length: {decompressedStream.Length}" );

                }  // end using.

             }  // end using.
          }

          /// <summary>
          ///
          /// </summary>
          public static MemoryStream Decompress( Stream compressedStream ) {

             compressedStream.Position = 0;

             MemoryStream decompressedStream = new MemoryStream();

             using ( DeflateStream decompressionStream = new DeflateStream( compressedStream, CompressionMode.Decompress ) ) {

                decompressionStream.CopyTo( decompressedStream );

             }  // end using.

             return decompressedStream;
          }

          /// <summary>
          ///
          /// </summary>
          public static MemoryStream Serialize( Person p ) {

             MemoryStream ms = new MemoryStream();
             BinaryFormatter formatter = new BinaryFormatter();

             formatter.Serialize( ms, p );

             return ms;
          }

       }  // end class.
    }


    GHS



    • Edited by bs777 Thursday, October 10, 2019 7:52 PM
    Thursday, October 10, 2019 7:38 PM
  • Hi,

    When I use the Serialize in UWP and .NET Standard 2.0, the stream I get is indeed different. But I'm not sure it is an expected behavior or an unexpected behavior. It's better to use compression and decompression only in UWP or .NET Standard 2.0. And I will invite other engineers to participate it.

    Best Regards,

    Fay


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Friday, October 11, 2019 8:43 AM
  • Fay,

    I've gone through the same processes with the XmlSerializer and the bottom line is:

    Compression and decompression (using DeflateStream or GzipStream) of "serialized" content DOES NOT WORK regardless of the platform.

    Microsoft should document it as such so others don't waste their time.

    And based on your response, it would be wise to "invite other engineers" if you're "not sure".

    Gerry


    GHS



    • Edited by bs777 Friday, October 11, 2019 1:54 PM
    Friday, October 11, 2019 1:51 PM
  • Yep, they should have MSDN article saying that grumpy Gary forgot to flush buffers of his compressionStream :)

    MCSD App Builder, MCSA UWP

    Saturday, October 12, 2019 10:07 AM
  • Hi,

    >>The source to the compression are binary serialized C# data "objects" that get deserialized at run time. The serialization and deserialization work fine (from Standard to UWP); it's the decompression from Standard to UWP that's an issue (when that step is added).

    If I understand correctly, your concern is that when for example object compressed in Standard Library and decompressed in UWP project. The object size changed. 

    The reason is the class is not the same, because the Object class defined in .Net standard and .Net Core is different. You can see that by F12 the Object class in two platforms:

    .Net Standard Object Class Source:

     .Net Core(UWP) Object Class Source:

    .Net Standard as a "standard" is a API cross-section subset of .Net Core .Net and Mono. UWP leverages the .Net Core, therefore it is not possible for them to have the same "Object" class.

    So,back to the problem, the best practise is to keep the compression and decompression process in the same platform to avoid this kind of issue.

    Hope it works for you,

    Elvis Xia


    Tuesday, October 15, 2019 6:22 AM
  • It's not an "object" that's being compressed; it's a "serialized stream" of the object; i.e. "binary data".

    If people would bother to run my "sample code" you would see that for that test I use "one" library; the problem persists.

    And for those that keep harping about "flushing", it makes no difference; I can track all lengths and positioning in debug.

    In fact, "flush" is not even implement for the compression stream:

          //
          // Summary:
          //     The current implementation of this method has no functionality.
          //
          // Exceptions:
          //   T:System.ObjectDisposedException:
          //     The stream is closed.
          public override void Flush();

    "Grumpy Gerry" did not just fall off the turnip truck; he gets grumpy when people just spout to the lowest denominator.


    GHS




    • Edited by bs777 Thursday, October 17, 2019 5:22 AM
    Thursday, October 17, 2019 5:12 AM
  • As I posted below, "flush" is not implemented.

    Run the sample code before you start spouting again. Make all the changes you want, THEN start talking.

    I can run all  serializing and compression functions in isolation; it's combining that doesn't work.


    GHS


    • Edited by bs777 Thursday, October 17, 2019 5:17 AM
    Thursday, October 17, 2019 5:13 AM
  • Hi,

    I might have misunderstood something. I tried again to compress the object in the .Net standard library and decompress it in the UWP project. And it works fine. I also checked with the size of MemoryStream. It's also the same.

    Here is the repo that I have made from your codes: CompressionSample. You can take a try.

    About decompressedStream.Length:

    DeflateStream.Length is not a supported attribute, you can F12 to see below info:

            // Summary:
            //     This property is not supported and always throws a System.NotSupportedException.
            //
            // Returns:
            //     A long value.
            //
            // Exceptions:
            //   T:System.NotSupportedException:
            //     This property is not supported on this stream.
            public override long Length { get; }

    Thanks,

    Elvis Xia



    Thursday, October 17, 2019 7:27 AM
  • Elvis,

    Thanks for the code.

    Your sample seems to work; made any changes to my sample; and still no luck.

    A couple of things: your size is 194, mine is 198, then 0.

    Your mixing x86 and any CPU, while I was targeting X64.

    You have 2 warnings that I don't (not aware of any suppression). I'm thinking I'm missing a setting. I'll let you know what I find.

    Also, the MS doc samples don't show flush, you use flush, I flush, no effect.

    [LATER]

    Copied my sample "code" to your solution into a new project; now it works (198,159,198). I also get the same warnings. Setting? Possible corrupt project?

    [LATER2]

    A new console app (with my code) targeting NET CORE 3.0 instead of NET Framework 4.7.2 (and NET Standard 2.0?) works. The clue was creating a new console app in your solution.

    [LAST]

    Flush is implemented in NET CORE 3.0, but not in NET Framework 4.7.2 (and the docs still suck in this area).

    In summary:

    This scenario works with Standard Lib 2.0 when called from Net Core 3.0 or UWP; it does NOT work when called from NET Framework 4.7.2.

    Just using Framework 4.7.2 does NOT work.

    Net Framework 4.7.2 is NOT 100% compatible with Net Standard 2.0

    Thanks Elvis.


    GHS








    • Edited by bs777 Thursday, October 17, 2019 6:02 PM
    Thursday, October 17, 2019 9:17 AM