System.OutOfMemory Exception

Question

I have Win7 Ultimate x64, and 32 GB RAM. I am using visual studio 2010 for development, c# language.

I have the following line in my program:

double[,] matrix = new double[n, n];

where n = 30000.

I have 32GB of RAM and I calculated it would be enought memory for a matrix of that size.

Thanks,

Aleksandar

Answer 1

Hi,

Doing a pseudo-calculation: Your array occupies at least (30,000^2)*8 bytes, which is about 6.70 GB. If you compiled your application as a 32bit application, the OOM exception is very logical, because you can't use an address space of 32 GB in a 32bit application.

The total address space limit per application is 4 GB, but at least 1 GB out of the 4 GB is reserved by the operating system.

Virtual Address Space
http://msdn.microsoft.com/en-gb/library/aa366912.aspx

Marcel

Answer 2

Hi, 

"Out of Memory" does't mean your physical memory size. You can see good article on this subject from 

http://blogs.msdn.com/b/ericlippert/archive/2009/06/08/out-of-memory-does-not-refer-to-physical-memory.aspx

How does I, even know before writing code is enough memory available?

For this purpose System.Runtime.MemoryFailPoint  class will come into rescue. You can check prior to assigning ragged array whether required memory can be allocated?, like 

         try  
           {   //checking for 1GB of memory availability  
               using (var memoryFailPoint = new MemoryFailPoint(1000))  
               {  
                   //perform memory consuming operation  
               }  
           }  
           catch (InsufficientMemoryException exception)  
           {  
               //This happens when there is not enough memory  
           }  

I hope this helps you..

---

If this post answers your question, please click "Mark As Answer". If this post is helpful please click "Mark as Helpful".

Answer 3

Hi Nevin,

Could you please describe the reason for unproposing my posting? This could be beneficial for future readers of this thread. By the way: Have you tried to run  Aleksandar'c code on a 32bit machine (and please don't just declare the array ;-) Did you succeed?

The kind of page swapping mentioned by Eric Lippert in the blog article cited by Kris444 does not apply to the storing of the above array. It could apply to the streaming of a very large data structure, where one does not need the whole data, after it was once processed, but it does not apply to .NET array. And it would be very hard to implement on a .NET basis. When there's no more place for storing on the heap for the array's items, an OOM exception is thrown. It's as simple as that.

You don't have to believe me. Just fire up VS or WinDbg, load SOS and take a close look at your heap and the stats.

Marcel

Answer 4

>>> Could you please describe the reason for unproposing my posting? This could be beneficial for future readers of this thread.

Sorry! An error on my part. :)

 

>>> By the way: Have you tried to run  Aleksandar'c code on a 32bit machine (and please don't just declare the array ;-) Did you succeed?

Unfortunately, I only have a 64-bit machine, and when I tried to run it I also got a System.OutOfMemoryException, as shown below:

 

>>> The kind of page swapping mentioned by Eric Lippert in the blog article cited by Kris444 does not apply to the storing of the above array. It could apply to the streaming of a very large data structure, where one does not need the whole data, after it was once processed, but it does not apply to .NET array. And it would be very hard to implement on a .NET basis. When there's no more place for storing on the heap for the array's items, an OOM exception is thrown. It's as simple as that.

Agreed.

 

Storing an array of that size is quite an interesting operation, it would be interesting to know exactly why you're doing that, Aleksandar :)

If I can be of any more help, let me know! :)

---

Nevin Janzen (Visit my Website)
If this post answers your question, please click Propose As Answer. If this post is helpful, please click Vote As Helpful.
How to Get Better Answers to Your Questions
P.S. My replies are not critical, but humble responses. :)

Answer 5

Hi Nevin,

> I only have a 64-bit machine, and when I tried to run it I also got a System.OutOfMemoryException

That shows clearly that we are not only running out of addressable virtual memory space. Arrays are object types, they also have well-known size constraints (theoretically 2GB, in practice far less). Even on 64-bit systems, where it should be possible to find a contiguous memory hole of that size, the maximum object size allowed in the GC Heap is still 2GB.

There's an excelent article on the subject written by Josh Williams (a former member in the CLR-Team):
BigArray<T>, getting around the 2GB array size limit

Marcel

Answer 6

Hi Marcel,

Nice article you are pointing to (BigArray<T>)..

OP is pointing to existing RAM size, and on this Eric's post is explaining OOM on behalf of physical memory. In comment section he mentioned on 2GB usermode memory limit (can go with additional GB) with 32bit (I do agree the article does not point to array size)

and here is post from Maoni , from GC team, on 32 bit vs 64 bit 

http://blogs.msdn.com/b/maoni/archive/2007/05/15/64-bit-vs-32-bit.aspx

and we can control gc allocation limit from .Net 4.5 by specifying

<gcAllowVeryLargeObects> to true, to allow specifying array > 2GB.

---

If this post answers your question, please click "Mark As Answer". If this post is helpful please click "Mark as Helpful".