locked
Out of memory exception RRS feed

  • Question

  • Hi,

    We get "out of memory exception" while loading/saving data by calling COM interop.

    Environment
    1. Windows 7 Ultimate (64 bit - Service Pack 1)
    2. RAM - 8GB, Processor - i7 - 3.40GHz
    2. Client - C# Winforms (.NET FRAMEWORK 2.0 - 32 bit)
    3. Server - Visual C++

    Note:
    This exception happens by calling a Interop through C# DotNet application (If the complus dll is called through VB6 exe, there is no memory exception).
    During save process, if we send above 150MB of data, this exception happens.

    Question:
    How can we overcome this memory issue with .Net application?

    Best regards,
    Suresh

    Saturday, December 1, 2012 2:27 PM

Answers

  • Try the following:

          //Out of memory exception
          //GC.Collect();
          //testProj.SaveDataEx(longText);
    
          // replace with the following
          // ptr may need to be cast appropriately
          var gch = GCHandle.Alloc(longText);
          var ptr = GCHandle.ToIntPtr(gch);
          testProj.SaveDataEx(ptr);

    gch.Free();


    "Premature optimization is the root of all evil." - Knuth

    If I provoked thought, please click the green arrow

    If I provoked Aha! please click Propose as Answer

    We are here to learn, to share knowledge, and to earn points; all in about equal measure.


    • Edited by Pieter Geerkens Thursday, December 6, 2012 7:11 PM Almost forgot gch.Free()
    • Proposed as answer by Bob Shen Monday, December 17, 2012 2:59 AM
    • Marked as answer by Bob Shen Wednesday, December 19, 2012 3:32 AM
    Thursday, December 6, 2012 7:09 PM

All replies

  • Here are some facts about the out of memmory exception for C#:

    http://blogs.msdn.com/b/ericlippert/archive/2009/06/08/out-of-memory-does-not-refer-to-physical-memory.aspx

    You could try running the Garbage Collector before the save:

    GC.Collect()


    "Premature optimization is the root of all evil." - Knuth

    If I provoked thought, please click the green arrow

    If I provoked Aha! please click Propose as Answer

    Saturday, December 1, 2012 2:40 PM
  • Hi Peter,

    Thanks for the information. It will be good if we can get some practical hints to overcome this issue through code (GC.Collect() etc).

    Architecture:

    STANDALONE

    C# Client <-> COM+ Interop <-> COM+ (VC++ 6.0) <-> DataBase (SQL Server)

    CLIENT

    C# Client <-> ASP.NET WebService <-> COM+ Interop <-> COM+ (VC++ 6.0) <-> DataBase (SQL Server)

    Sample Scenario:

    Word document is stored as blob data in the database.

    1. Blob data is fetched from database and passed to client as byte array.

    2. Byte array is converted as word document and shown in the interface.

    3. Similar strategy is followed, for saving the data.

    Question:

    If the word document is huge, does it mean that we have to load/send the byte array data in parts from/to the COM+ & subsequently to database? (it will be good, if there is some other better solution)

    Note:

    GC.Collect() doesn't solve the issue

    Rgds, Suresh

    Tuesday, December 4, 2012 1:48 PM
  • It is difficult to conceive of a Word document that would so large as to make that necessary.

    Remember that all COM and COM+ data structures are unmanaged, and so must be explicitly deallocated when no longer needed. Also, all data structures passed in to COM or COM+ objects mustbe pinned sothat the Garbae Collector does not attempt to move it while inuse by the unmanaged code.


    "Premature optimization is the root of all evil." - Knuth

    If I provoked thought, please click the green arrow

    If I provoked Aha! please click Propose as Answer

    Tuesday, December 4, 2012 2:08 PM
  • This issue has been reported by some of our customers & it is easy to reproduce (open word document, copy some big images to it and ensure the size is > 75 MB).

    We have taken care in releasing the unmanaged code & this issue happens only with .NET to COM+ interop. Also, we have a Legacy application which uses VB6.0. It works without any issue with huge data (VB6 to COM+ - the same client code has been migrated to C#).

    Tuesday, December 4, 2012 2:35 PM
  • Sounds like GC is moving storage around while it is in use by COM+. Check that you are pinning all managed data structures passed in to COM+ interop while they are in use by unmanaged code.

    "Premature optimization is the root of all evil." - Knuth

    If I provoked thought, please click the green arrow

    If I provoked Aha! please click Propose as Answer

    Tuesday, December 4, 2012 4:14 PM
  • C# Code

    private byte[] docArray; private void button1_Click(object sender, EventArgs e) { try { docArray = ReadByteArrayFromFile(Application.StartupPath + \\test.doc); //VB6 var testProj = new TestProject.clsSave(); var recCnt = 1; var longTextData = new object[3, recCnt]; for (var i = 0; i < recCnt; i++) longTextData[1, i] = docArray; object longText = longTextData; //Works testProj.SaveDataEx(longText); recCnt = 10; longTextData = new object[3, recCnt]; for (var i = 0; i < recCnt; i++) longTextData[1, i] = docArray; longText = longTextData; //Out of memory exception //GC.Collect(); testProj.SaveDataEx(longText); } catch (Exception ex) { MessageBox.Show(ex.Message + " " + ex.StackTrace); } } private static bool IsFileExists(string path) { return File.Exists(path); } private static byte[] ReadByteArrayFromFile(string filePath) { byte[] byteDoc = null; if (IsFileExists(filePath)) { try { byteDoc = File.ReadAllBytes(filePath); } catch (Exception){} } return byteDoc; }

    VB6

    Public Function SaveDataEx(byteData As Variant) As String
    On Error GoTo ErrHandler

        MsgBox "ENTERED"
        SaveDataEx = "END"
       
    ErrHandler:
        If Err.Number <> 0 Then MsgBox Err.Number & " " & Err.Description
    End Function

    It is easy to reproduce the exception with the above sample code.

    Note:

    Interop has to be generated for the VB6 Component and reference it in C# application (ActiveX dll).

    Create a word document with roughly 140 MB of data.

    Thursday, December 6, 2012 8:44 AM
  • I'm stepping back to the top of the thread in case you send more code.

    It looks like you are encountering a race-condition of some sort. Your small (single record) example  is working because it is fast, while your long (10-record) example is slower and encounters the problem. You are NOT pinning memory before calling:

    testProj.SaveDataEx(longText);

    As mentioned in an earlier post, pinning your managed object is necessary before passing it to unmanaged code. Check these two references:
    http://msdn.microsoft.com/en-us/library/system.runtime.interopservices.gchandle.aspx
    http://msdn.microsoft.com/en-us/library/23acw07k(v=vs.100).aspx


    "Premature optimization is the root of all evil." - Knuth

    If I provoked thought, please click the green arrow

    If I provoked Aha! please click Propose as Answer

    We are here to learn, to share knowledge, and to earn points; all in about equal measure.

    Thursday, December 6, 2012 9:54 AM
  • Pieter, Could you let me know how the sample code needs to be changed?
    Thursday, December 6, 2012 4:30 PM
  • Try the following:

          //Out of memory exception
          //GC.Collect();
          //testProj.SaveDataEx(longText);
    
          // replace with the following
          // ptr may need to be cast appropriately
          var gch = GCHandle.Alloc(longText);
          var ptr = GCHandle.ToIntPtr(gch);
          testProj.SaveDataEx(ptr);

    gch.Free();


    "Premature optimization is the root of all evil." - Knuth

    If I provoked thought, please click the green arrow

    If I provoked Aha! please click Propose as Answer

    We are here to learn, to share knowledge, and to earn points; all in about equal measure.


    • Edited by Pieter Geerkens Thursday, December 6, 2012 7:11 PM Almost forgot gch.Free()
    • Proposed as answer by Bob Shen Monday, December 17, 2012 2:59 AM
    • Marked as answer by Bob Shen Wednesday, December 19, 2012 3:32 AM
    Thursday, December 6, 2012 7:09 PM