locked
Out of Memory Exceptions while running Tests

    Question

  • I am having issues while running a test suite.  I get Out of Memory exceptions while running and was wondering if anything could be done to see if MSTest is actually cleaning up memory when a test is complete.  I have tried to add the following in my test suites:

     

    [TestCleanup]

    public void MyTestCleaup()

    {

    GC.Collect();

    GC.WaitForPendingFinalizers();

    }

     

    It seems to me that somewhere memory doesn't get cleaned up.

     

    Thanks,

     

    Monday, June 02, 2008 9:09 PM

Answers

  • I see. It is by design that our test framework does not ask GC to collect memory. But if you need, you can do that yourself by calling GC.Collect like you did in your sample code from TestCleanup. Note that ClassCleanup is not a good place because the time when we call it is not defined (in fact we will call it in the very end, right before we clean up for the entire test run).

     

    Could you make sure that TestCleanup is called for previous (before failed) test?

    You can also do some diagnostics tracing: before/after collecting memory in TestCleanup add something like

    Console.Write("{0:000000}k", GC.GetTotalMemory(false) / 1000);

    Then, in Test Results window, do add/remove columns, check StdOut and StartTime, then sort by StartTime. In this way you could see in the results grid, how much memory is left after each test is started.

    Or you can pause the test before and after call to GC and then in ResourceManager/Winiternals Process Explorer see if memory is disposed or not.

    Could the problem be that you left too small portion of memory for you test?

     

    Thank you,

    Michael

     

    P.S. For both test execution order and more deterministic way to call ClassCleanup: we consider improving this scenario in v.next. Would you like to vote for that? :-)

    Wednesday, June 04, 2008 9:38 PM

All replies

  • Hi Adam, we do not explicitly clean up memory while running tests, GC is doing that when it decides it's needed. OutOfMemoryException usually indicate to some test issue. Could you please provide call stack at the point when you get this exception to us? Also if you can capture memory dump (run under/attach vs debugger to vstesthost.exe, with unmanaged debugging enabled, then when you stop at OutOfMemoryException [1st chance, i.e. 1st time you get it], do: Debug->Save dump file as... -> minidump with heap).

     

    BTW, your code to call GC.Collect in ClassCleanup: did it help you? When you do that, do you see the OutOfMemoryException?

     

    Thank you,

    Michael Koltachev, Visual Studio Team Test

    Wednesday, June 04, 2008 7:56 AM
  • Michael,

     

    >> OutOfMemoryException usually indicate to some test issue. 

    I don't think that I explained my problem very well, so let me try to better explain.  We have written tests that are trying to test our memory limit.  Meaning we chew up lots of memory in our tests so that our functionality can run with minimal memory without throwing exceptions.  While running our tests we chew up memory in some tests but don't care about chewing up memory in other tests.  We need to be able to free the memory when we run each test but the GC doesn't seem to free the memory.

     

    >> Could you please provide call stack at the point when you get this exception to us?

    I am trying to do my best here but this is the first call stack that I got when I got the first OutOfMemoryException:

    --------------------------------------------------------------------------------------------------------------------------------------------------------

    System.OutOfMemoryException was caught
      Message="Exception of type 'System.OutOfMemoryException' was thrown."
      Source="Spiricon.DataSource.Tests"
      StackTrace:
           at Spiricon.DataSource.Tests.FrameBufferTest.TestRawFrame..ctor(Int32 x, Int32 y) in E:\Version5\LBA\Source\Spiricon.DataSource.Tests\FrameBufferTest.cs:line 164
           at Spiricon.DataSource.Tests.FrameBufferTest.GenerateFrames(Int32 rows, Int32 cols, Int32 num, IFrameBuffer buffer) in E:\Version5\LBA\Source\Spiricon.DataSource.Tests\FrameBufferTest.cs:line 260
           at Spiricon.DataSource.Tests.FrameBufferTest.MemLimitTest() in E:\Version5\LBA\Source\Spiricon.DataSource.Tests\FrameBufferTest.cs:line 947
      InnerException:
    --------------------------------------------------------------------------------------------------------------------------------------------------------

    Looking at the exception it looks like it is our test that is causing the problem.

     

    >> Also if you can capture memory dump (run under/attach vs debugger to vstesthost.exe, with

    >> unmanaged debugging enabled, then when you stop at OutOfMemoryException [1st chance, i.e.

    >> 1st time you get it], do: Debug->Save dump file as... -> minidump with heap).

     

    I tried to do the above and when I go to save the dump it doesn't give me the option.  I don't know if I am doing it wright but I will try to explain how I am doing this.  I am running on VS .Net 2008.  I enabled the "unmanaged debugging enabled" check box on each of the test projects property->debug tabs.  I then built my project and ran the tests.  Once the tests are running I attached to the VSTesthost.exe process but when the exception is thrown it doesn't break.  So I put a try catch block around the code that throws the exception.  I put a break point in the catch and when the exception is caught I tried to save the dump but the option didn't exist in the Debug tab.  I have the feeling this is a PEBKAC.  Any help on this issue would be great.

     

    Like I explained first we have tests that are pushing memory limits and the GC doesn't seem to clean up enough memory.  When we run the tests by themselves or with less tests they seem to run just fine.

     

    >> BTW, your code to call GC.Collect in ClassCleanup: did it help you? When you do that, do you see the

    >> OutOfMemoryException?

    Yes I still see it.

     

     

    Thanks

    Wednesday, June 04, 2008 8:11 PM
  • I see. It is by design that our test framework does not ask GC to collect memory. But if you need, you can do that yourself by calling GC.Collect like you did in your sample code from TestCleanup. Note that ClassCleanup is not a good place because the time when we call it is not defined (in fact we will call it in the very end, right before we clean up for the entire test run).

     

    Could you make sure that TestCleanup is called for previous (before failed) test?

    You can also do some diagnostics tracing: before/after collecting memory in TestCleanup add something like

    Console.Write("{0:000000}k", GC.GetTotalMemory(false) / 1000);

    Then, in Test Results window, do add/remove columns, check StdOut and StartTime, then sort by StartTime. In this way you could see in the results grid, how much memory is left after each test is started.

    Or you can pause the test before and after call to GC and then in ResourceManager/Winiternals Process Explorer see if memory is disposed or not.

    Could the problem be that you left too small portion of memory for you test?

     

    Thank you,

    Michael

     

    P.S. For both test execution order and more deterministic way to call ClassCleanup: we consider improving this scenario in v.next. Would you like to vote for that? :-)

    Wednesday, June 04, 2008 9:38 PM
  • Michael,

      Thanks for the reply's I am trying a few things from you last suggestions and I will get back to you.  I would vote for your suggestion in your P.S. remark.  I think that a more deterministic way to call ClassCleanup would be great!

     

    Adam

    Thursday, June 05, 2008 4:58 PM