locked
Simulating low memory for native C++ unit tests RRS feed

  • Question

  • Hi,

    I am writing native C++ code in Visual Studio 2012 and recently started to create unit tests (in Visual Studio). The code which I want to test calls the new operator, and I wrote code to handle memory allocation errors, but I have no idea how to test this error handler. Execution of unit tests is integrated into the company's build system, so I cannot simply allocate lots of memory and hope the unit test will run out of space - colleagues depending on the build system will kill me. What is more, I also want to test my code with enough memory available.

    To make the story short: Is there a way to tell my unit test project that for some tests a call to new has to fail? Something along the lines of _set_new_failure(1)?

    Others have already suggested that I write my own new operator but that would be unit testing under my new operator; what I want is to test under MS's new operator because this is going to be the real-life situation. Others yet say that new cannot fail, but I believe in Murphy's Law which states that anything that CAN go wrong WILL go wrong. As a matter of fact, I've seen much simpler code fail.

    Thank you

    Hans

    Friday, December 5, 2014 9:15 PM

All replies

  • The question is just where do you stop with error checking ?

    Do you then do error checking of the error checking.

    I have never bothered about error checking the new statement in 30 years of programming and it hasn't caused a problem.


    n.Wright

    Friday, December 5, 2014 9:37 PM
  • I have never bothered about error checking the new statement in 30 years of programming and it hasn't caused a problem.


    From other posts on the internet I know that new can fail. It is not too unlikely that some programmer will, some day, use my code in a situation where new will fail. (Murphy's Law again!) So I want to verify the behaviour of my program in that situation.

    I've seen error handling code that was never tested, and when the error occurred for the first time ever, the error handler crashed the program because of some silly bug. I don't want to end up with something like that.

    • Edited by H4N51 Saturday, December 6, 2014 1:16 PM
    Saturday, December 6, 2014 1:13 PM
  • Hi Hans,

    Thank you for posting in MSDN forum.

    >>Is there a way to tell my unit test project that for some tests a call to new has to fail? Something along the lines of _set_new_failure(1)?

    According to your description, I suggest you can refer the following article to create a unit test to test this error handler.

    Reference:

    http://www.codeproject.com/Articles/6108/Simulating-Memory-Allocation-Failure-for-Unit-Test

    >>I've seen error handling code that was never tested, and when the error occurred for the first time ever, the error handler crashed the program because of some silly bug

    You mean that you have been tried to create a unit test to call the new operator, but you get some error, am I right?

    If yes, please you share the code for me so that we will further help you solve this issue.

    Best Regards,


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.


    • Edited by Tina-Shi Monday, December 8, 2014 9:57 AM
    Monday, December 8, 2014 9:53 AM
  • Hi Tina-Shi, and thank you for your reply.

    >> According to your description, I suggest you can refer the following article to create a unit test to test this error handler.

    I came across this great article during my original research for this problem. It replaces the 'new' operator by one that fails in a controlled way. Of course it helpsto know that my code will gracefully clean up when my replacement 'new' fails - but it is not quite the same as to know that it can also handle a failure of the built-in 'new'... (I'm trying to run the unit tests under conditions as close to reallife as possible.)

    I had hoped there would be an instruction, a test method attribute, a command line switch - whatever, to tell the unit test that 'new' and 'malloc' it should fail. It is, by the way, also a question of code coverage in my unit tests.

    >>You mean that you have been tried to create a unit test to call the new operator, but you get some error, am I right?

    Ah sorry - I wasn't expressing myself properly. This was a different situation, and neither unit tests nor the new operator were involved. It was along these lines: To track an error situation which I guessed might occur now and then, I wrote a error handler that produced formatted log file output. When the error actually occurred, the program crashed because I had confused %d and %s in the format string. This comes from not testing the error handler.

    Best

    Hans

    Edit: I found this on http://msdn.microsoft.com/EN-US/library/kftdy56f%28d=hv.2,v=vs.120%29.aspx:

    In Visual C++ .NET 2002, the new function in the Standard C++ Library will support the behavior specified in the C++ standard, which is to throw a std::bad_alloc exception if the memory allocation fails.

    While this is rather straight, I found it isn't true for VS2012. If new fails to allocate enough memory, it throws a CMemoryException. I wonder whether the behaviour has changed over time and the docs weren#t updated, over whether MS forgot to mention that the behaviour of 'new' depends on circumstances.

    Finally, this is why I want to include the this scenario in the unit test!


    • Edited by H4N51 Tuesday, December 9, 2014 8:38 PM
    Monday, December 8, 2014 6:13 PM
  • Hi Hans,

    Glad to receive your reply.

    >> The new function in the Standard C++ Library will support the behavior specified in the C++ standard, which is to throw a std::bad_alloc exception if the memory allocation fails. I found it isn't true for VS2012. If new fails to allocate enough memory, it throws a CMemoryException.

    According to your description, could you please tell me what new function you write in your C++ code in VS2012?

    I did some research about the CMemoryException, I found that if you are writing a new function using MFC, it will throw the CMemoryException.

    Reference:

    http://msdn.microsoft.com/en-us/library/bc8b34bc.aspx

    However, if you new function in the Standard C++ Library, it will throw the std::bad_alloc exception.

    Therefore, the issue is not related to the MSDN document update.

    If possible, I suggest you’d better share the C++ code for me so that we will further support.

    In addition, as you said that you want to test MS's new operator using unit test to verify the memory allocation errors. I find a similar thread about the issue, it is mainly used to new Argument to test the new operator fail.

    Reference:

    http://stackoverflow.com/questions/12445729/testing-new-operator-fail

    Best Regards,


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.


    • Edited by Tina-Shi Wednesday, December 10, 2014 6:12 AM
    Wednesday, December 10, 2014 6:06 AM
  • Hi Tina-Shi, thanks for your message!

    The paragraph from my message which you quoted was a quotation from the MSDN docs, so it does not refer to a new function I wrote (neither to an 'operator new' function I wrote), but to the 'new' function aka '::operator new' built into the C++ language. Yes, it is true, my project uses MFC. I assume MFC redefines '::operator new' in a way that it throws CMemory exception rather than bad_alloc, probably via _set_new_handler.

    I already knew the link to stackoverflow. It shows how to define an '::operator new' that fails in a user-defined way. My unit test would then verify that my code behaves well when the user-defined 'new' fails. This is not what I want to test because it can never happen in real life. In real life, the predefined 'new' fails... rarely, I admit, but more often than never...

    As a preliminary measure, I pass a memory size into 'new' that must fail, but once we produce 64-bit code, this may not be enough.

    Regards

    Hans

    Sunday, December 14, 2014 9:50 AM
  • Hi Hans,

    Glad to receive your reply.

    >> As a preliminary measure, I pass a memory size into 'new' that must fail, but once we produce 64-bit code, this may not be enough.

    Since I am not C++ experts, I think the issue is more related to the C++ programming, so I suggest you can ask the issue to C++ develop forum about the 'new' function aka '::operator new', it will bebetter support.

    Thanks for your understanding.

    Best Regards,


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Monday, December 15, 2014 10:48 AM
  • Hi,

    did I express myself so unclear? I was asking about writing a unit test that would make the 'new' operator fail, in order to test my program's error handler. Answer #1 was along the lines, if you have an error handler, you've done enough, testing it would be overkill. The rest of this thread was mostly trying to clear up misunderstandings.

    No, I was not asking how to overload or replace the 'new' operator. I was not talking about writing code to allocate memory, nor about extending MFC. Admittedly, my question was C++ specific, but what should one expect from a question about unit tests for C++? I'll switch to the C++ forum, but I bet they will say "your questions are about unit tests, please go to the unit tests forum."

    Maybe I'd better just give up on this :-(

    Hans

    Monday, December 15, 2014 7:17 PM