Out of memory exception

Question

Hi,

We get "out of memory exception" while loading/saving data by calling COM interop.

Environment
1. Windows 7 Ultimate (64 bit - Service Pack 1)
2. RAM - 8GB, Processor - i7 - 3.40GHz
2. Client - C# Winforms (.NET FRAMEWORK 2.0 - 32 bit)
3. Server - Visual C++

Note:
This exception happens by calling a Interop through C# DotNet application (If the complus dll is called through VB6 exe, there is no memory exception).
During save process, if we send above 150MB of data, this exception happens.

Question:
How can we overcome this memory issue with .Net application?

Best regards,
Suresh

Answer 1

Here are some facts about the out of memmory exception for C#:

http://blogs.msdn.com/b/ericlippert/archive/2009/06/08/out-of-memory-does-not-refer-to-physical-memory.aspx

You could try running the Garbage Collector before the save:

GC.Collect()

---

"Premature optimization is the root of all evil." - Knuth

If I provoked thought, please click the green arrow

If I provoked Aha! please click Propose as Answer

Answer 2

Hi Peter,

Thanks for the information. It will be good if we can get some practical hints to overcome this issue through code (GC.Collect() etc).

Architecture:

STANDALONE

C# Client <-> COM+ Interop <-> COM+ (VC++ 6.0) <-> DataBase (SQL Server)

CLIENT

C# Client <-> ASP.NET WebService <-> COM+ Interop <-> COM+ (VC++ 6.0) <-> DataBase (SQL Server)

Sample Scenario:

Word document is stored as blob data in the database.

1. Blob data is fetched from database and passed to client as byte array.

2. Byte array is converted as word document and shown in the interface.

3. Similar strategy is followed, for saving the data.

Question:

If the word document is huge, does it mean that we have to load/send the byte array data in parts from/to the COM+ & subsequently to database? (it will be good, if there is some other better solution)

Note:

GC.Collect() doesn't solve the issue

Rgds, Suresh

Answer 3

It is difficult to conceive of a Word document that would so large as to make that necessary.

Remember that all COM and COM+ data structures are unmanaged, and so must be explicitly deallocated when no longer needed. Also, all data structures passed in to COM or COM+ objects mustbe pinned sothat the Garbae Collector does not attempt to move it while inuse by the unmanaged code.

---

"Premature optimization is the root of all evil." - Knuth

If I provoked thought, please click the green arrow

If I provoked Aha! please click Propose as Answer

Answer 4

This issue has been reported by some of our customers & it is easy to reproduce (open word document, copy some big images to it and ensure the size is > 75 MB).

We have taken care in releasing the unmanaged code & this issue happens only with .NET to COM+ interop. Also, we have a Legacy application which uses VB6.0. It works without any issue with huge data (VB6 to COM+ - the same client code has been migrated to C#).

Answer 5

Sounds like GC is moving storage around while it is in use by COM+. Check that you are pinning all managed data structures passed in to COM+ interop while they are in use by unmanaged code.

---

"Premature optimization is the root of all evil." - Knuth

If I provoked thought, please click the green arrow

If I provoked Aha! please click Propose as Answer

Answer 6

C# Code
private byte[] docArray;

        private void button1_Click(object sender, EventArgs e)
        {
            try
            {
                docArray = ReadByteArrayFromFile(Application.StartupPath + \\test.doc);

                //VB6
                var testProj = new TestProject.clsSave();

                var recCnt = 1;
                var longTextData = new object[3, recCnt];
                for (var i = 0; i < recCnt; i++)
                    longTextData[1, i] = docArray;

                object longText = longTextData;
                
                //Works
                 testProj.SaveDataEx(longText);

                recCnt = 10;
                longTextData = new object[3, recCnt];
                for (var i = 0; i < recCnt; i++)
                    longTextData[1, i] = docArray;

                longText = longTextData;

                //Out of memory exception
                //GC.Collect();
                testProj.SaveDataEx(longText);
                }
            catch (Exception ex)
            {
                MessageBox.Show(ex.Message + " " + ex.StackTrace);
            }
        }
        private static bool IsFileExists(string path)
        {
            return File.Exists(path);
        }
        private static byte[] ReadByteArrayFromFile(string filePath)
        {
            byte[] byteDoc = null;
            if (IsFileExists(filePath))
            {
                try
                {
                    byteDoc = File.ReadAllBytes(filePath);
                }
                catch (Exception){}
            }
            return byteDoc;
        }

VB6

Public Function SaveDataEx(byteData As Variant) As String
On Error GoTo ErrHandler

    MsgBox "ENTERED"
    SaveDataEx = "END"
   
ErrHandler:
    If Err.Number <> 0 Then MsgBox Err.Number & " " & Err.Description
End Function

It is easy to reproduce the exception with the above sample code.

Note:

Interop has to be generated for the VB6 Component and reference it in C# application (ActiveX dll).

Create a word document with roughly 140 MB of data.

Answer 7

I'm stepping back to the top of the thread in case you send more code.

It looks like you are encountering a race-condition of some sort. Your small (single record) example  is working because it is fast, while your long (10-record) example is slower and encounters the problem. You are NOT pinning memory before calling:

testProj.SaveDataEx(longText);

As mentioned in an earlier post, pinning your managed object is necessary before passing it to unmanaged code. Check these two references:
http://msdn.microsoft.com/en-us/library/system.runtime.interopservices.gchandle.aspx
http://msdn.microsoft.com/en-us/library/23acw07k(v=vs.100).aspx

---

"Premature optimization is the root of all evil." - Knuth

If I provoked thought, please click the green arrow

If I provoked Aha! please click Propose as Answer

We are here to learn, to share knowledge, and to earn points; all in about equal measure.

Answer 8

Pieter, Could you let me know how the sample code needs to be changed?

Answer 9

Try the following:

      //Out of memory exception
      //GC.Collect();
      //testProj.SaveDataEx(longText);

      // replace with the following
      // ptr may need to be cast appropriately
      var gch = GCHandle.Alloc(longText);
      var ptr = GCHandle.ToIntPtr(gch);
      testProj.SaveDataEx(ptr);

      gch.Free();

---

"Premature optimization is the root of all evil." - Knuth

If I provoked thought, please click the green arrow

If I provoked Aha! please click Propose as Answer

We are here to learn, to share knowledge, and to earn points; all in about equal measure.