locked
XElement.Save() Out of memory exception RRS feed

  • Question

  • Hi All,

    I am facing OOM exception when I am Saving XElement object using below function.

    Any idea how we can avoid the exception?

    private static byte[] Serialize(IEnumerable<MyEntity> entity)
            {
                var element = ToXml(entity);  //Return XElement object
                using (var stream = new MemoryStream())
                {
    //element.Save(Stream);  //OOM
                    element.Save(stream, SaveOptions.DisableFormatting);  //OOM
                    return stream.ToArray();
                }
               
            }

    Thanks,

    Ajit

    • Moved by Mike Danes Tuesday, June 18, 2013 8:09 AM not WPF related
    Tuesday, June 18, 2013 5:06 AM

Answers

  • Hi,

    I have solved this issue with compressing the XML file.

    Thanks,

    Ajit

    • Marked as answer by Mike Feng Tuesday, June 25, 2013 5:23 AM
    Thursday, June 20, 2013 11:40 AM

All replies

  • Heh, more out of memory errors. 2 questions:

    1. how big is the xml?
    2. does this happen in the same "RenderTargetBitmap" program?
    Tuesday, June 18, 2013 5:41 AM
  • Hi Mike,

    Yes, This is second time OOM exception I found.

    The Size of the xml is big.. Ihavn't check the exact size but I have 1575 entity.

    That is not same as "RenderTargetBitmap" :). This is with other workflow.

    -Ajit

    Tuesday, June 18, 2013 6:28 AM
  • 1575 entities doesn't sound much but perhaps each entity is large. Anyway, using MemoryStream to write large amounts of data tends to be a bad idea. It keeps reallocating an array and when the capacity of the stream gets large that may cause problems with the large object heap. The fact that you also need to do ToArray doesn't help.

    Best thing would be to use temporary files. If that's not possible then try setting the capacity of the MemoryStream (before writing to it) to a value that's large enough to accomodate the xml.

    PS: this has nothing to do with WPF, I'll move it to the BCL forums since ultimately it's about using MemoryStream.

    Tuesday, June 18, 2013 6:38 AM
  • Hi Mike,

    The xml file size is 120 MB. Do you think it will cause the OOM issue?

    -Ajit

    Tuesday, June 18, 2013 9:11 AM
  • At least on x86 yes. If you do this repeatedly it's possible to end up with a fragmented large object heap that may be unable to allocate a single block of 120MB. And if you keep around multiple resulting byte arrays then it will be even worse.
    Tuesday, June 18, 2013 9:20 AM
  • Okie...just now I tried with the tempFile instead of MemoryStream and it works. 

    Do you have any other better option that to use tempFile?

    -Ajit

    Tuesday, June 18, 2013 9:48 AM
  • Have you tried to set the MemoryStream capacity? Set it to 128 megabytes for example, that should avoid large object heap fragmentation.

    Tuesday, June 18, 2013 9:53 AM
  • Hi,

    The MemoryStream takes the buffer size on "int". So maxmum it can take int.max size.

    I tried to change code as beow:

       using (var stream = new MemoryStream(2147483647))
                {
                    element.Save(stream, SaveOptions.DisableFormatting);

               }

    on bold line it gives OOM again...

    -Ajit


    Tuesday, June 18, 2013 10:34 AM
  • "So maxmum it can take int.max size."

    And I told you to use 128 megabytes (134217728), not int.MaxValue. With int.MaxValue you'll end up allocating 2GB of memory...

    Tuesday, June 18, 2013 11:11 AM
  • I tried with 128MB still it's having out of memory exception... what is the max size that we can give to MemoryStream()??

    -Ajit

    Tuesday, June 18, 2013 11:24 AM
  • "what is the max size that we can give to MemoryStream()??"

    In theory it's around 2 GB. In practice on x86 the max size is much lower due to the limited address space and fragmentation.

    Tuesday, June 18, 2013 11:29 AM
  • Hi,

    I have solved this issue with compressing the XML file.

    Thanks,

    Ajit

    • Marked as answer by Mike Feng Tuesday, June 25, 2013 5:23 AM
    Thursday, June 20, 2013 11:40 AM