Failed to allocate 128 Mb with 64 bits application

Question

Hi,

I'm on a Windows 7 64 bits platform and I have a 64 bits application. I try to allocate 134217728 bytes (128 Mb) but it returns me a 0x0000000.. pointer !

How is it possible ? and what can I do ?

Thanks

---

PL01

Answer 1

Please show us the code which you are using which displays this behaviour.  Also, which version of Visual C++ are you using?

---

Answering policy: see profile.

Answer 2

You could start by showing us the code.

Answer 3

It is with Visual Studio 2010.

The code is simple :

        malloc(newSize);

In the task manager, it tells that 1GB is already used.

Thanks

---

PL01

Answer 4

My crystal ball says you have an error on line 42 of your source.  Maybe it is because you did not assign the value returned my malloc to a pointer.

If you new what the error was, you would not be asking the question.  Show us a compilable block of code which produces the unexpected results.

Answer 5

How many memory has been installed on your computer ?

How many free memory is enable on your computer ?

Could you give the complete code arround your " malloc(newSize);"

---

Delphine GARRO

Answer 6

Computer total memory 16Gb
Computer used memory is 6.4 Gb
Software used memory 933 Mb

About the code, it is just a malloc... nothing special... but in general there are several part of the code that allocate memory, by example I use QT for the UI, I use OpenCL that handle its own memory and I have my own memory manager to allocate memory.

What I would like to know is when a ::malloc can return 0x000000 in 64 even if it remain a lot of memory ?

Thanks

---

PL01

Answer 7

Are you absolutely sure this is an x64 application? Check again. This looks like the behaviour of a 32-bit application.

Answer 8

Yes,

I compile it in 64 bits... in VS2010 I have create the 64 bits configuration version from the 32 bits version (a copy). It seems that VS changed it to 64 bits automatically !

When I open it with DepedencyWalker... it tell me it is 64 bits and all the DLLs too :-(

---

PL01

Answer 9

>on a Windows 7 64 bits platform and ... a 64 bits
>application. ... allocate 134217728 bytes (128 Mb)
>... returns a 0x0000000 pointer

>How is it possible?

Common causes:

(1) Insufficient free heap to satisfy the request.

(2) Insufficient *unfragmented* heap free to satisfy
    the request.
    
(3) Heap corruption. Prior operations have corrupted
    the heap or the heap manager code, such as: doing
    a free() with an invalid pointer or a pointer
    that has already been freed; overwriting part of
    the heap or heap manager code by buffer overruns
    or using wild/invalid pointers; overwriting the
    heap control block at the start of an allocation
    area with the result that a later free or realloc
    of that allocation uses invalid information re the
    heap; etc.

>what can I do?

Debug your program.
    
>I compile it in 64 bits...

You can verify this in the IDE by checking the type for
which size_t is an alias. In a 32-bit build it will be a
typedef for an unsigned int. In a 64-bit build it will
be a typedef for an unsigned __int64. Hovering the mouse
cursor over "size_t" in the editor should pop up a tooltip
bubble showing the typedef. Also, in a 64-bit build
sizeof(size_t) will be 8 while in a 32-bit build it will
be 4.
        
- Wayne

Answer 10

Thanks

And how can I debug this kind of problem ? There is no tool to check the memory fragmentation !

---

PL01

Answer 11

>There is no tool to check the memory fragmentation !

If it has in fact been built as a 64-bit program
then it is unlikely that heap fragmentation is the
cause of the malloc failure. Look for problems as
described under item (3).

- Wayne

Answer 12

Thanks,

In fact, to debug this I have do the following, to check if I don't free some memory zone several times or try to free invalid zones.

	#define MemoryAlloc(A) pureCore::Alloc::DebugAlloc(__FILE__, __LINE__, (A))
	#define MemoryFree(A) pureCore::Alloc::DebugFree(__FILE__, __LINE__, (A))

	inline void*    operator new        (size_t size)       { return pureCore::Alloc::DebugAlloc(__FILE__, __LINE__, size); }
	inline void*    operator new[]      (size_t size)       { return pureCore::Alloc::DebugAlloc(__FILE__, __LINE__, size); }
	inline void     operator delete     (void* ptr)         { return pureCore::Alloc::DebugFree(__FILE__, __LINE__, ptr); }
	inline void     operator delete[]   (void* ptr)         { return pureCore::Alloc::DebugFree(__FILE__, __LINE__, ptr); }

#define ALLOC_HEADER_CHECKSUM 123456789
#define ALLOC_HEADER_CHECKSUM_FREE 987654321

struct AllocHeader
{
    AllocHeader*    prev;
    AllocHeader*    next;
    size_t          size;
	unsigned int	checksum;			// Allow to test memory corruption
};

static AllocHeader                      _memAllocs         = { &_memAllocs, &_memAllocs, 0, ALLOC_HEADER_CHECKSUM };

void* pureCore::Alloc::DebugAlloc(const char* file, int line, size_t size)
{
    assert(size >= 0);

    _lock.enter();

    AllocHeader* alloc = (AllocHeader*)::malloc(sizeof(AllocHeader) + size);
    if (!alloc)
        throw pureCore::MemoryException("Out of memory!");

    void* ptr           = alloc + 1;
    alloc->prev         = _memAllocs.prev;
    alloc->next         = &_memAllocs;
    alloc->prev->next   = alloc;
    alloc->next->prev   = alloc;
    alloc->size         = size;
	alloc->checksum		= ALLOC_HEADER_CHECKSUM;
    _memoryUsed        += size; // += _msize(ptr) - sizeof(AllocHeader);

    _lock.leave();

    return ptr;
}

void pureCore::Alloc::DebugFree(const char* file, int line, void* ptr)
{
    if (!ptr)
        return;

    _lock.enter();

	AllocHeader* alloc = (AllocHeader*)ptr - 1;

	if (alloc->checksum	== ALLOC_HEADER_CHECKSUM_FREE)
		throw pureCore::MemoryException("Memory already freed!");
	if (alloc->checksum	!= ALLOC_HEADER_CHECKSUM)
		throw pureCore::MemoryException("Invalid memory freed!");
    
    alloc->prev->next = alloc->next;
    alloc->next->prev = alloc->prev;
	alloc->checksum		= ALLOC_HEADER_CHECKSUM_FREE;
    _memoryUsed -= alloc->size;
    ::free(alloc);

    _lock.leave();
}

But unfortunately it seems that there is no error of this way !

---

PL01

Answer 13

In fact, in this debug method, I do the following :

1) I check that I don't free some memory several times

2) I check that I don't free a wrong memory zone

3) I check that some memory zones are not corrupted

... and to summarize, this code find no error !!!!!

So, what to do ?

---

PL01

Answer 14

This is a shot in the dark, but what is SIZE_MAX set to? (It should be _UI64_MAX, which should be 0xffffffffffffffff, but I don't always trust the windows header files/Predefined MACROS) If it is only UINT_MAX than _WIN64 is probably not being set, causing all sorts of problems.

Also, what runtime libraries are you using?

Answer 15

Thanks,

But _WIN64 is set :-(

For info, I use VC++ 2010. I'm still searching why my malloc failed, but can't find any solution !

---

PL01

Answer 16

Which runtime libraries are you using (v90 or v100 (found in Properties (General) and the Runtime library flag (Code Generation (should be /MT for application, /MD for a dll..) These things being messed up/mismatched could cause your problems.

Answer 17

I use v100.

My application is build with /MD (like all my DLLs) , I have supposed that it is just the way to link to the runtime library !?

What I understand from here : http://msdn.microsoft.com/en-us/library/2kzt1wy3(v=vs.80).aspx is that it is not a problem to link with the runtime as a DLL !

---

PL01

Answer 18

Until you actually provide a sample which shows this behaviour, you won't manage to get anyone to magically guess at what your problem is.

The code that you did post above is useless since it doesn't show the circumstances that it is being used under and it also doesn't deal with the possibility of heap corruption properly. (You do realise that simply writing one extra element in an array will cause heap corruption that this class of your can't detect.)

So give your code, or a small sample that replicates the behaviour.

---

This is a signature

Any samples given are not meant to have error checking or show best practices. They are meant to just illustrate a point. I may also give inefficient code or introduce some problems to discourage copy/paste coding. This is because the major point of my posts is to aid in the learning process.

Do you want Visual Studio 11 Express to be freely installable on Windows 7 and able to write regular C++ applications? Please vote for this.

Answer 19

My application is hughe and really complex.... so, if I was able to reproduce the problem in a small sample I will be able to find the problem maybe. But the problem is that I have no idea of the problem reason and no idea where the problem happend !!!

I'm trying to fix this since 2 month now... no success !!! Also, it is a commercial application and I cannot post the 400 Mb of source code !

:-( Sorry...

---

PL01

Answer 20

What is the purpose of your application ?

Why do you not try to start to a little program that only allocate 128MB memory, and add, step by step, all your librairy to identify which causse default allocation. And so perhaps could you find a solution.

We have'nt enough informations to help you...

---

Delphine GARRO

Answer 21

Hi,

My application is here http://www.spectralpixel.com

The problem is that "malloc" does not report why it failed !

---

PL01

Answer 22

>... in general there are several part of the code that
>allocate memory, by example I use QT for the UI, I use
>OpenCL that handle its own memory and I have my own
>memory manager to allocate memory.

Any of the above may be the source of heap corruption,
leaks, or heap exhaustion/fragmentation resulting in a
failed malloc.

>I'm trying to fix this since 2 month now... no success !!!
>Also, it is a commercial application and I cannot post the
>400 Mb of source code !

Since it is a commercial program, it is worth investing in
3rd-party debugging software. Have you tried analyzing your
program with diagnostic software such as these?

Rational Purify for Windows
http://www-01.ibm.com/software/awdtools/purify/win/

Rational PurifyPlus for Windows
http://www-01.ibm.com/software/awdtools/purifyplus/win/

DevPartner
http://www.microfocus.com/products/micro-focus-developer/devpartner/index.aspx

AQtime Pro Performance Profiling
Profiling in Microsoft Visual Studio
http://smartbear.com/products/qa-tools/application-performance-profiling/profiling-in-microsoft-visual-studio

PC-lint for C/C++
http://www.gimpel.com/html/index.htm

- Wayne