locked
Get a not valid heap pointer when using new RRS feed

  • Question

  • Hello everyone,

    the first time I am posting a question in msdn, and I hope this it the correct forum for the question.

    The question (actually big helpless problem) is : I got a not valid heap pointer when using the new operator.

    The situation is like this : I have many structures to save different information (I can tell that I am working on reading the PE format, so you may know what the structures are used for), somewhere inside one structure I use "new" to create an array of objects of another structure. First I got the assert failure of _CrtIsValidHeapPointer when I delete the array of objects in the destructor. Then I searched the internet and found some possible causes. The only possible cause I think is double deleting the array. But the thing is I do not find any place that the array is deleted before the last call of the destructor (I read the code and also debugged, this point should be sure). Then I found out I can validate the pointer using _CrtIsValidHeapPointer, so I put _CrtIsValidHeapPointer just after the "new", supprisingly it fails!

    I may show some similar code:

    structA.member = new MemberOfStructA[structA.numberOfMembers];

    _CrtIsValidHeapPointe(structA.member);

    This call of _CrtIsValidHeapPointer fails. I think you know what I mean, the assert fails, more precisely, the HeapValidate function in _CrtIsValidHeapPointer fails, but I cannot debug into the HeapValidate function, so I don't know what happens there.

    I think now it should be clear, it is not the problem of deleting the array twice, the problem is why I get a not valid heap pointer from "new"?! The strange thing is, this failure only happens by this "new", it does not happen in other "new"s (I tested for the other "new"s).

    Any suggestion or walkaround would be nice, but please don't say something that is out of the topic (I saw many out-of-topic-solutions in other forums).

    PS: some of the my project settings are as followed:

    One console exe and one dll, the class of the dll is exported using an interface, and the exe loads the dll using dynamic loading (LoadLibrary), the object of the class in the dll is a global one, it is "newed" and deleted via the two function calls which are exported, so the new and delete are done inside the dll (or dll heap).  The runtime library used for the exe and the dll is multi threaded debug dll. No managed code is used.  One more thing to mention is that the heap is corrupted also in release mode.

    Wednesday, March 9, 2011 4:04 PM

Answers

  • Hmm, you're digging too much into this _CrtIsValidHeapPointer thing. Let's make one thing clear:

    If _CrtIsValidHeapPointer returns false when called on a pointer returned by new you don't necesarilly have a problem. _CrtIsValidHeapPointer is intended to work with malloc, in many cases it will work with new too but that's not required and it's not a bug.

    Now, back to your real problem, why does delete assert? If we exclude double deletes, unmatched new[]/delete[] and other obvious mistakes we're left with one thing: heap corruption. How does heap corruption happen? Simply, you allocate 100 bytes and write 101 bytes.

    PS: I see that you have a non virtual destructor. Do you inherit from that class? If so you should make the desctructor virtual.

    • Marked as answer by edwardcarlfox Thursday, March 10, 2011 1:15 PM
    Thursday, March 10, 2011 11:16 AM

All replies

  • When you use new[] the returned pointer doesn't always point to the begining of the allocated memory. That's because the runtime needs to store the number of allocated elements somewhere so when you delete the array it know how many objects to destruct.

    If that's what happens here then your problem is likely caused by using delete instead of delete[]. Or that you're trying to delete the allocated memory through a pointer of a different type, like void *.

     

    Wednesday, March 9, 2011 4:17 PM
  • When you use new[] the returned pointer doesn't always point to the begining of the allocated memory. That's because the runtime needs to store the number of allocated elements somewhere so when you delete the array it know how many objects to destruct.

    If that's what happens here then your problem is likely caused by using delete instead of delete[]. Or that you're trying to delete the allocated memory through a pointer of a different type, like void *.

     


    Thanks for the reply Mike!

    According to the possible causes you mentioned, first, I use definitely delete[], and second, I don't understand what you mean by deleting the memory through a pointer of a different type. So here is a similar destructor:

    ~StructA()

    {

            delete[] member;

    }

    A very simple one. The only thing is I put this definition in the header file, so just inside the definition of the structure. But I don't think this is a big issue.

    Regarding to the first paragraph you said, I know what you are talking about, there are 4 more bytes just before the first object in the array, I saw this in the memory window when debugging. And I also noticed that when destructing each object the pointer to each object is correct, and when deleting the array itself, the pointer is 4 bytes less then the pointer of the first object (the 4 bytes information). And also the object are deleted from the end to the beginning, so deleting the objects are fine, but just after deleting the first object (the last to be deleted) the assert fails because it is not a valid heap pointer. So I think the problem goes back to, why I got already a not valid heap pointer from "new".

    Anyway thanks again, Mike!

    Wednesday, March 9, 2011 4:28 PM
  • "So I think the problem goes back to, why I got already a not valid heap pointer from "new"."

    Nope. There are 2 different problems here:

    1. first I got the assert failure of _CrtIsValidHeapPointer when I delete the array of objects in the destructor
    2. why I got already a not valid heap pointer from "new"

    2. can be explained by new[]'s behavior. Try something like _CrtIsValidHeapPointe((size_t *)structA.member - 1). If this fails too... I'm afraid that your code is corrupting the heap.

    But 1. can be caused by many things. Corrupted memory, double deletes, delete[] instead of delete and so on.

    "what you mean by deleting the memory through a pointer of a different type"

    Something like this:

    void *p = new foo[10];
    delete[] p;
    Wednesday, March 9, 2011 4:39 PM
  • Concerning the two different problems you mentioned Mike, the first one, I did not use delete instead of delete[], up to now I do not find any other place that the array is deleted other than the one in the destructor, and in debugging I found out that the destructor is called at the end of the program. And you said corrupted memory, what is it supposed to mean? How the memory going corrupted automatically?

    The second one, I still don't get it. I know that there is 4 more bytes just before the allocated memory, and as you said, the runtime knows how many objects to delete, yes, I found the number in that 4 bytes is 9, and actually I have 9 elements (in the example I am working on). So all these are clear. And I don't think that because of the new[] behaviour I will have a invalid heap pointer, all the other "new"s work, but this one does not. One little thing I just discovered is, this new creates an array of objects of user defined structure, but the others are basic data types. But I also don't think this is the problem, since I have also other dynamic allocated arrays consisting of user defined objects, but only they are not creating because of the the type of the example I am using (The exmaple I mentioned here is the different types of binary files, like dll, lib and so on).

    And you said I try _CrtIsValidHeapPointer((size_t*)structA.member - 1), what is the meaning of this call? Do you mean I use the pointer to the previous 4 bytes instead of the pointer from "new"?

    Thursday, March 10, 2011 9:05 AM
  • Try something like _CrtIsValidHeapPointe((size_t *)structA.member - 1).

    Mike, I just tried what you suggested, tried _CrtIsValidHeapPointer((size_t*)structA.member - 1), and this call passed! But it still failed when deleting the array, well of course, since the pointer used in deleting the array is still the one that points to the beginning of the array, not the previous 4 bytes. So now I am even more confused, any help?
    Thursday, March 10, 2011 9:12 AM
  • Ok everyone, I went deeper into the problem, and I found some strange things.

    First through debugging, I found out that actually there are 4 more bytes just before the beginning of the allocated memory, but VS will put the value of these 4 bytes to 0xfdfdfdfd, while the actual memory allocated is 0xcdcdcdcd. Although I don't know how this 0xfdfdfdfd helps, but it is there.

    Then, when I use _CrtIsValidHeapPointer to validate the pointer, the actual value that is passed to the function is the pointer I get from the "new", that is the value of the pointer pointing to the beginning of the memory, not the previous 4 bytes.

    Ok, that means if I got a invalid heap pointer from "new", then the value of the pointer indeed does not point to the beginning of the heap. I found out this by debugging into the "new" operator, and normally the size of memory is just the same as the size of the total bytes needed, and I think the extra 4 bytes are allocated automatically by the heap manager or somewhere else, at least I cannot see it. But when I debugged into the "new" where I got the problem, FOUR EXTRA BYTES ARE ADDED ALREADY BEFORE THE CALL OF THE "NEW" OPERATOR!!! What the xxxx! I confirmed this by going into the assembly codes, and indeed, before other "new" operators, everything is normal, and finally I got memory with 4 extra bytes, but before the problem "new", these 4 bytes are added explicitly (showed in assembly), so further another 4 bytes are added in the heap manager, finally I got 8 more bytes! And the pointer I got points to the position 4bytes after the beginning of the memory! This is just wrong, and causes the problem!

    Now I am wondering if this is a compiler error or what, I cannot control the creation of the assembly or machine codes, is it a bug in the compiler?

    Here I can show the assembly with the source codes of two functions I used to compare, the assembly up to the call of the new[]:

    1. The new that works:

    sectionHeaders = new SectionHeader[coffHeader.numberOfSections];

        mov    eax, DWORD PTR _this$[ebp]
        movzx    eax, WORD PTR [eax+52]
        xor    ecx, ecx
        mov    edx, 84                    ; 00000054H
        mul    edx
        seto    cl
        neg    ecx
        or    ecx, eax
        push    ecx
        call    ??_U@YAPAXI@Z                ; operator new[]


    2. The new that fails:

    archive.objMembers = new ArchiveObjMember[archive.secondLinkerMember.numberOfMembers];

        mov    eax, DWORD PTR _this$[ebp]
        mov    ecx, DWORD PTR [eax+700]
        mov    DWORD PTR $T8328[ebp], ecx
        xor    ecx, ecx
        mov    eax, DWORD PTR $T8328[ebp]
        mov    edx, 184                ; 000000b8H
        mul    edx
        seto    cl
        neg    ecx
        or    ecx, eax
        xor    eax, eax
        add    ecx, 4
        setb    al
        neg    eax
        or    eax, ecx
        push    eax
        call    ??_U@YAPAXI@Z                ; operator new[]

    Pay attention to the bold italic and underscored instruction, this is the evil!!!

    So someone has any suggestion?

    Thursday, March 10, 2011 10:21 AM
  • Hmm, you're digging too much into this _CrtIsValidHeapPointer thing. Let's make one thing clear:

    If _CrtIsValidHeapPointer returns false when called on a pointer returned by new you don't necesarilly have a problem. _CrtIsValidHeapPointer is intended to work with malloc, in many cases it will work with new too but that's not required and it's not a bug.

    Now, back to your real problem, why does delete assert? If we exclude double deletes, unmatched new[]/delete[] and other obvious mistakes we're left with one thing: heap corruption. How does heap corruption happen? Simply, you allocate 100 bytes and write 101 bytes.

    PS: I see that you have a non virtual destructor. Do you inherit from that class? If so you should make the desctructor virtual.

    • Marked as answer by edwardcarlfox Thursday, March 10, 2011 1:15 PM
    Thursday, March 10, 2011 11:16 AM
  • Thanks again Mike,

    actually I agree with you, there should be no problem with just using new and delete. And I hope the problem is caused by me, so I will examine the codes more carefully and try to find out the actual problem.

    And I actually don't have a virtual destructor, since this is only a structure to pack all information that I get from reading the file. And FYI I have many other structures even without a constructor and destructor, I have destructor here only to delete dynamic allocated memory, and of course a constructor to initialize the pointers to null.

    Besides, I am really confused by the behaviour of the assembly and the memory, just don't know why there are 8 more bytes in this "new", and the second 4 bytes are 0x09000000, which is also the number of the elements in the array, and this kind of value does not exist in other "new"s though.

    Anyway, I hope the problem is my codes, and it causes some strange behaviour, hope I could find a solution soon.

    Thursday, March 10, 2011 12:31 PM
  • Mike, I feel pathetic but I solved the problem finally! It turned out that I did overwrite the heap! I have an array of numbers and when reading the file I fill the array, but the number of elements is unknown, and I did not use dynamic allocation, I defined a big enough max number for the array. The problem is not the big enought number, it is big enough, but I want to set the rest elements which are not used to something equivalent to null, and here I did not use the predefined big enough number, I used some bigger number for the for loop! That is why I overwrite the heap, and this array is at the end of the heap I mentioned above.

    Oh my god, thank you very much Mike!I will remember what you said, "Simply, you allocate 100 bytes and write 101 bytes."

    Thursday, March 10, 2011 1:15 PM
  • "I have an array of numbers and when reading the file I fill the array, but the number of elements is unknown, and I did not use dynamic allocation, I defined a big enough max number for the array"

    Maybe you should use std::array (http://msdn.microsoft.com/en-us/library/bb983093.aspx). When running in debug mode it checks if the index is within bounds :)

    Thursday, March 10, 2011 1:28 PM
  • Thanks for the hint Mike, actually I want to do stuff in a more native manner, and I know many basic stuff are not safe, but I want to earn a good programming manner and thought through doing so unsafe stuff, without relying on other things that can guard me, and I can get better and be trained by doing so. (Hope you can understand what I said, I am a chinese, not so professional in English:-))

    Thursday, March 10, 2011 3:55 PM