locked
'System.OutOfMemoryException'

    Question

  • Guys,

    I am running an application in debug and it continuously falls over because of a memory exception.  I am creating all my objects locally and nulling them once I have finished.  But form some reason, after several loops it falls over at the same point where I create a new object[].
    When I dimensionise the array it is in the region of 400,000 in size.

    I was under the impression C# took care of its own garbage collection? 

    If not, what do I need to do, to make sure I have enough memory available to complete the task?


    The code is:

    public ArrayList DataAmalgamator(ArrayList myCcyList)
            {
                ArrayList MasterList = new ArrayList();
                ArrayList TradeList = new ArrayList();
                ArrayList QuoteList = new ArrayList();
                ArrayList ScreenedList = new ArrayList();
                War_TradeSim.mDataEvent[] myDataEvent;
                int i = 0;
                int counterAmount = ElementCounter(myCcyList);
                myDataEvent = new mDataEvent[counterAmount];//THIS IS WHERE THE ERROR OCCURS  'System.OutOfMemoryException' 
                counterAmount = 0;
                foreach (ArrayList al in myCcyList)
                {
                    War_TradeSim.mQuote[] mq = new mQuote[] { };
                    War_TradeSim.mQuote_Screened[] ms = new mQuote_Screened[] { };
                    War_TradeSim.mTrade[] mt = new mTrade[] { };
                    Object[] TheObj = new Object[] { };
                    Object[] TheObj1 = new Object[] { };
                    Object[] TheObj2 = new Object[] { };
                    Object[] TheObj3 = new Object[] { };
                    TheObj = (Object[])al[0];
                    ms = (War_TradeSim.mQuote_Screened[])TheObj[0];
                    mq = (War_TradeSim.mQuote[])TheObj[1];
                    mt = (War_TradeSim.mTrade[])TheObj[2];
                    #region
                    if (mt != null)
                    {
                        foreach (mTrade mti in mt)
                        {
                            myDataEvent[i].BTexchange = mti.BTexchange;
                            myDataEvent[i].Price = mti.Price;
                            if (mti.Time == 0.0)
                            {
                                break;
                            }
                            myDataEvent[i].Time = mti.Time;
                            myDataEvent[i].Dir = mti.Dir;
                            myDataEvent[i].exchange = mti.exchange;
                            myDataEvent[i].ccyName = mti.ccyName;
                            myDataEvent[i].type = "Trade";
                            if (mti.ccyName == "AUD")
                                myDataEvent[i].Inst = 0;
                            if (mti.ccyName == "CAD")
                                myDataEvent[i].Inst = 1;
                            if (mti.ccyName == "CHF")
                                myDataEvent[i].Inst = 2;
                            if (mti.ccyName == "EUR")
                                myDataEvent[i].Inst = 3;
                            if (mti.ccyName == "EURCHF")
                                myDataEvent[i].Inst = 4;
                            if (mti.ccyName == "EURGBP")
                                myDataEvent[i].Inst = 5;
                            if (mti.ccyName == "EURJPY")
                                myDataEvent[i].Inst = 6;
                            if (mti.ccyName == "EURNOK")
                                myDataEvent[i].Inst = 7;
                            if (mti.ccyName == "EURSEK")
                                myDataEvent[i].Inst = 8;
                            if (mti.ccyName == "GBP")
                                myDataEvent[i].Inst = 7;
                            if (mti.ccyName == "JPY")
                                myDataEvent[i].Inst = 8;
                            if (mti.ccyName == "MXN")
                                myDataEvent[i].Inst = 11;
                            if (mti.ccyName == "NZD")
                                myDataEvent[i].Inst = 9;
                            if (mti.ccyName == "ZAR")
                                myDataEvent[i].Inst = 13;
                            if (mti.ccyName == "XAU")
                                myDataEvent[i].Inst = 14;
                            if (mti.ccyName == "EURPLN")
                                myDataEvent[i].Inst = 15;
                            if (mti.ccyName == "EURHUF")
                                myDataEvent[i].Inst = 16;
                            if (mti.ccyName == "EURCZK")
                                myDataEvent[i].Inst = 17;
                            i++;
                        }
                        mt = null;
                    }
                    if (mq != null)
                    {
                        foreach (mQuote mqi in mq)
                        {
                            myDataEvent[i].Time = mqi.Time;
                            myDataEvent[i].BTexchange = mqi.BTexchange;
                            myDataEvent[i].Ask = mqi.Ask;
                            myDataEvent[i].AskSize = mqi.AskSize;
                            myDataEvent[i].Bid = mqi.Bid;
                            myDataEvent[i].BidSize = mqi.BidSize;
                            myDataEvent[i].ccyName = mqi.ccyName;
                            myDataEvent[i].exchange = mqi.exchange;
                            myDataEvent[i].type = "Quote";
                            if (mqi.ccyName == "AUD")
                                myDataEvent[i].Inst = 0;
                            if (mqi.ccyName == "CAD")
                                myDataEvent[i].Inst = 1;
                            if (mqi.ccyName == "CHF")
                                myDataEvent[i].Inst = 2;
                            if (mqi.ccyName == "EUR")
                                myDataEvent[i].Inst = 3;
                            if (mqi.ccyName == "EURCHF")
                                myDataEvent[i].Inst = 4;
                            if (mqi.ccyName == "EURGBP")
                                myDataEvent[i].Inst = 5;
                            if (mqi.ccyName == "EURJPY")
                                myDataEvent[i].Inst = 6;
                            if (mqi.ccyName == "EURNOK")
                                myDataEvent[i].Inst = 7;
                            if (mqi.ccyName == "EURSEK")
                                myDataEvent[i].Inst = 8;
                            if (mqi.ccyName == "GBP")
                                myDataEvent[i].Inst = 7;
                            if (mqi.ccyName == "JPY")
                                myDataEvent[i].Inst = 8;
                            if (mqi.ccyName == "MXN")
                                myDataEvent[i].Inst = 11;
                            if (mqi.ccyName == "NZD")
                                myDataEvent[i].Inst = 9;
                            if (mqi.ccyName == "ZAR")
                                myDataEvent[i].Inst = 13;
                            if (mqi.ccyName == "XAU")
                                myDataEvent[i].Inst = 14;
                            if (mqi.ccyName == "EURPLN")
                                myDataEvent[i].Inst = 15;
                            if (mqi.ccyName == "EURHUF")
                                myDataEvent[i].Inst = 16;
                            if (mqi.ccyName == "EURCZK")
                                myDataEvent[i].Inst = 17;
                            i++;
                        }
                        mq = null;
                    }
                    if (ms != null)
                    {
                        foreach (mQuote_Screened msi in ms)
                        {
                            myDataEvent[i].Time= msi.Time;
                            myDataEvent[i].BTexchange = msi.BTexchange;
                            myDataEvent[i].Ask = msi.Ask;
                            myDataEvent[i].AskSize = msi.AskSize;
                            myDataEvent[i].Bid = msi.Bid;
                            myDataEvent[i].BidSize = msi.BidSize;
                            myDataEvent[i].ccyName = msi.ccyName;
                            myDataEvent[i].exchange = msi.exchange;
                            myDataEvent[i].type = "QuoteScreened";
                            if (msi.ccyName == "AUD")
                                myDataEvent[i].Inst = 0;
                            if (msi.ccyName == "CAD")
                                myDataEvent[i].Inst = 1;
                            if (msi.ccyName == "CHF")
                                myDataEvent[i].Inst = 2;
                            if (msi.ccyName == "EUR")
                                myDataEvent[i].Inst = 3;
                            if (msi.ccyName == "EURCHF")
                                myDataEvent[i].Inst = 4;
                            if (msi.ccyName == "EURGBP")
                                myDataEvent[i].Inst = 5;
                            if (msi.ccyName == "EURJPY")
                                myDataEvent[i].Inst = 6;
                            if (msi.ccyName == "EURNOK")
                                myDataEvent[i].Inst = 7;
                            if (msi.ccyName == "EURSEK")
                                myDataEvent[i].Inst = 8;
                            if (msi.ccyName == "GBP")
                                myDataEvent[i].Inst = 7;
                            if (msi.ccyName == "JPY")
                                myDataEvent[i].Inst = 8;
                            if (msi.ccyName == "MXN")
                                myDataEvent[i].Inst = 11;
                            if (msi.ccyName == "NZD")
                                myDataEvent[i].Inst = 9;
                            if (msi.ccyName == "ZAR")
                                myDataEvent[i].Inst = 13;
                            if (msi.ccyName == "XAU")
                                myDataEvent[i].Inst = 14;
                            if (msi.ccyName == "EURPLN")
                                myDataEvent[i].Inst = 15;
                            if (msi.ccyName == "EURHUF")
                                myDataEvent[i].Inst = 16;
                            if (msi.ccyName == "EURCZK")
                                myDataEvent[i].Inst = 17;
                            i++;
                        }
                        ms = null;
                    }
                }
                    #endregion
                ArrayList TheList = new ArrayList();
                ArrayList mdeList = new ArrayList();
                foreach (mDataEvent mde in myDataEvent)
                {
                    MasterList.Add(mde);
                }
                myDataEvent = null;
                MasterList.Sort();
                TheList = FilterMasterList(MasterList);
                TheList.Sort();
                return TheList;
            }


    Thanks,
    Harry
    Thursday, November 06, 2008 10:56 AM

Answers

All replies

  • Hi Harry,

    I've noticed, you add all elements from myDataEvent array to MasterList. MasterList then passes filtered elements to TheList.

    DataAmalgamator passes this objects outside of the method... The problem is not in the DataAmalgamator method, but somewhere outside.

    Some external object must keep references to filtered elements from TheList.

    Vitaliy Liptchinsky
    Thursday, November 06, 2008 12:24 PM
  • Hi Dude - So you think the problem could be outside DataAmalgamator();?

    I use the TheList = FilterMasterList(MasterList); to filter my data before sorting it!  How could that have an impact on a memory exception?

    Thanks.
    H
    Thursday, November 06, 2008 1:27 PM
  • You've mentioned that after several loops you get OutOfMemoryException. Have you tried to check number of elements returned each time by DataAmalgamator?
    All filtered out elements inside DataAmalgamator are destroyed withing next garbage collection.

    Have you tried to call GC.Collect() after each DataAmalgamator call?

    By the way, which .NET version do you use? (There are some issues with large objects in .NET 1.0)


    Vitaliy Liptchinsky
    Thursday, November 06, 2008 2:11 PM
  • Hi Dude - I am using V2.0.5 SP1.

    I hadn't used GC.Collect, i was under the impression C# had its own garbage collection?  I guess I use GC.Collect after the callback for DataAmalgamator();

    Cheers
    Harry
    Thursday, November 06, 2008 2:23 PM
  • Hi Vitaliy - I added the GC.Collect(); but now it falls over in another part of the application.  Do you know of a way to diagnose where I could have a memory leak?

    Thanks,
    Harry

    Thursday, November 06, 2008 2:50 PM
  • Yes, .NET has it's own garbage collector and in 99.99% of cases you should not call GC.Collect() manually. (Don't tell anybody, I've made you use GC.Collect() :):):) )

    But in some circumstances, especially when you play with very large objects, it is recommended to call it.

    Vitaliy Liptchinsky
    Thursday, November 06, 2008 2:51 PM
  • Try memory profilers to diagnose the problem.
    Red Gate's memory profiler or CLRProfiler

    Vitaliy Liptchinsky
    Thursday, November 06, 2008 4:17 PM