Accessing List<T> items without inintializing all previous items

Question

Hi,

I have an object  defined below:

List<DocumentMetaData> myObject = new List<DocumentMetaData>();

Where:

 [Serializable]
    public class DocumentMetaData
    {        
        public List<NewCompactSimilar > morphological_occuerences = new List<NewCompactSimilar>();   
        public List<NewCompactSimilar> subClasses_occurences = new List<NewCompactSimilar>();       
        public List<NewCompactSimilar> superClasses_occurences = new List<NewCompactSimilar>();    
        public List<NewCompactSimilar> synonyms_occuerences = new List<NewCompactSimilar>();      
        public List<NewCompactSimilar> has_instances_occuerences = new List<NewCompactSimilar>(); 
        public List<NewCompactSimilar> instance_of_occuerences = new List<NewCompactSimilar>();    

    }

 [Serializable]
    public class NewCompactSimilar
    {
        public List<int> offsets;
        public List<String> words;
        public int pageID;

        public NewCompactSimilar()
        {
            offsets = new List<int>();
            words = new List<string>();          
        }
}

 

'myObject' working as a two dimensional array, but not all of its slots will be filled. Only specific cells , represented by the NewCompactSimilar objects, may get values during processing.

However I need to keep all of them te determine the correct index (virtual raw and column) of the data.

However, in order to fill the item 100 in 

List<NewCompactSimilar > morphological_occuerences

I need to initialize all of the 99 items before, even if they are empty or not really used . That causes the 'OutOfMemory' problem to be occurred with the large number of data.

How could I solve that?

Thank you so much.

Regards,

Aya.

---

Aya Zoghby

Answer 1

Huh? What "two dimensional array"? You're only working with List<T>..

When it is about memory problems in such a kind of structure, then you should consider using either Lazy<T> where appropriate or implement the proxy pattern.

Lazy<T> initializes the instances when first accessing them, thus you may end in a complete object graph. Also resulting in a memory problem.

Using the proxy pattern means, that you initialize for each object a proxy (placeholder). As this proxy object should have a smaller memory foot print, this can avoid the memory problem.

Using the factory pattern/strategy pattern combination or a different object graph could also be a solution. But this strongly depends on your context (use-case).

Answer 2

Thank you so much for the reply.

But I am using .Net Framework 3.5, so how could I use it ?

---

Aya Zoghby

Answer 3

Then you can implement lazy properties. E.g.

    public class DocumentMetaData
    {
        private List<NewCompactSimilar> morphologicalOccuerences = null;

        public List<NewCompactSimilar> MorphologicalOccuerences
        {
            get
            {
                if (this.morphologicalOccuerences == null)
                {
                    this.morphologicalOccuerences = new List<NewCompactSimilar>();
                }

                return this.morphologicalOccuerences;
            }
        }
    }

Answer 4

However, in order to fill the item 100 in 

List<NewCompactSimilar > morphological_occuerences

I need to initialize all of the 99 items before, even if they are empty or not really used . That causes the 'OutOfMemory' problem to be occurred with the large number of data.

Have you tried like this :

for(int i =0;i<98;i++){

morphological_occurences.Add(null); }

morphological_occurences.Add(new NewCompactSimilar);

Since , for 99 cell you want no objects so , Add null . Wereever you need object , Add it .

Hope this helps.

---

One good question is equivalent to ten best answers.

Answer 5

However, in order to fill the item 100 in 

List<NewCompactSimilar > morphological_occuerences

I need to initialize all of the 99 items before, even if they are empty or not really used . That causes the 'OutOfMemory' problem to be occurred with the large number of data.

Have you tried like this :

for(int i =0;i<98;i++){

morphological_occurences.Add(null); }

morphological_occurences.Add(new NewCompactSimilar);

Since , for 99 cell you want no objects so , Add null . Wereever you need object , Add it .

Hope this helps.

---

One good question is equivalent to ten best answers.

Thank you for the reply, however that also yields 'OutOfMemory'!!!

It was working when I used ArrayLists before using List<> and filling items by .add(0)!!

---

Aya Zoghby

Answer 6

Then you can implement lazy properties. E.g.

    public class DocumentMetaData
    {
        private List<NewCompactSimilar> morphologicalOccuerences = null;

        public List<NewCompactSimilar> MorphologicalOccuerences
        {
            get
            {
                if (this.morphologicalOccuerences == null)
                {
                    this.morphologicalOccuerences = new List<NewCompactSimilar>();
                }

                return this.morphologicalOccuerences;
            }
        }
    }

Thank you so much, but it also yields an 'OutOfMemory' !!

The only worked way was when I used an ArrayList instead of List and using .add(0) !!

But I won't use the ArrayLists any more, so what could I do.

---

Aya Zoghby

Answer 7

Hi,

try this tool : http://memprofiler.com/

---

One good question is equivalent to ten best answers.

Answer 8

PLEASE NOTE THATH:

The 'OutOfMemory' Message appeares when I try to serialize the object 'myObject'.

---

Aya Zoghby

Answer 9

PLEASE NOTE THATH:

The 'OutOfMemory' Message appeares when I try to serialize the object 'myObject'.

---

Aya Zoghby

This may help :

http://stackoverflow.com/questions/709399/how-to-serialize-big-objects-in-net-outofmemory-exceptions

---

One good question is equivalent to ten best answers.

Answer 10

You could use a Dictionary<int, NewCompactSimilar> if you want to be able to set any item at any index.

Answer 11

I'm just curious on why you just didn't use the actual two dimensional array? It seems like it would be easier and possibly less overhead / memory usage.

Thanks,

Brad

Answer 12

I'm just curious on why you just didn't use the actual two dimensional array? It seems like it would be easier and possibly less overhead / memory usage.

Beyond some threshold, a Dictionary with only one item with a key defined as X uses less memory than an array of X items. I didn't check its exact value but I think that threshold is rather low.

Answer 13

On average you can assume that a dictionary will be around half full, so if you have a table that's less than half populated you'll *save* memory using a Dictionary (on average).