none
Why not create a thread, --- LastStatusValue: (NTSTATUS) 0xc0000017

    Question

  • Excuse me, need your help urgently.

    I'm testing an application which is a Corba server based on OMNIORB 4.1.6 running on Windows 7. It is developed on VS2008. During this stress test, some Corba client applications will do some operations randomly, such as connect to Corba server, send requests to Corba server, disconnect from Corba server. After running about two weeks, then Corba server application will fail to create thread by calling function _beginthreadex. The last error is as follows:

    LastErrorValue: (Win32) 0x8 (8) - Not enough storage is available to process this command.

    LastStatusValue: (NTSTATUS) 0xc0000017 - {Not Enough Quota}  Not enough virtual memory or paging file quota is available to complete the specified operation.

    For my knowledge, when Corba client send a request to server application, the server application will create a thread to service the request. After completion, the thread will be recycled.

    After the failure, I could create more than 100 threads in other application by calling the function “_beginthreadex”.

    Furthermore, for the server application, running as "Administrator", the memory used is 36M, the thread count is 633, the handle count is 1535.  For the whole system, the CPU, resource is normal as ever.

    To my strange, after the first failure, when I restart the Corba server application again, the failure could be reproduced quite soon, maybe need 3-8 hours.

    The question has almost driven me crazy, could you give me some advice?

    Thanks

    Thursday, November 1, 2012 5:48 AM

Answers

  • "After the failure, I could create more than 100 threads --in other application--- by calling the function “_beginthreadex”. "

    Creating threads in other application does not necessarily indicate that the server application doesn't have a problem. For example on x86 a particular process may fail to create additional threads because the address space (limited to 2GB on x86) is too fragmented and it cannot allocate space for the thread stack (by default 1MB). But since each process has its own address space another process on the same computer may create additional threads.

    Ultimately you may want to ask OMNIORB developers, they may be able too tell you if having 633 threads is expected, sounds a bit too much.

    • Marked as answer by xinsong001 Thursday, November 1, 2012 9:40 AM
    Thursday, November 1, 2012 8:16 AM
    Moderator
  • Sorry to tell you my result early.

    After investigation, I found the large unusable memory is due to an anti-virus software I installed, the software seems monitor and tag the memory allocation of every user mode process.


    Friday, November 9, 2012 10:23 AM

All replies

  • "After the failure, I could create more than 100 threads --in other application--- by calling the function “_beginthreadex”. "

    Creating threads in other application does not necessarily indicate that the server application doesn't have a problem. For example on x86 a particular process may fail to create additional threads because the address space (limited to 2GB on x86) is too fragmented and it cannot allocate space for the thread stack (by default 1MB). But since each process has its own address space another process on the same computer may create additional threads.

    Ultimately you may want to ask OMNIORB developers, they may be able too tell you if having 633 threads is expected, sounds a bit too much.

    • Marked as answer by xinsong001 Thursday, November 1, 2012 9:40 AM
    Thursday, November 1, 2012 8:16 AM
    Moderator
  • Great, your judgement is right.

    my colleague found that many unusable memory exist, amount to 1.2G, and usable memory is less than 64K. So it should not create the thread.

    However, I wonder why it could reproduce the problem quite soon after I restart the Corba server application without restarted the windows system. it only need 3-8 hours, instead of two weeks.

    Thanks again



    • Edited by xinsong001 Thursday, November 1, 2012 9:39 AM
    Thursday, November 1, 2012 9:16 AM
  • That does sound a bit odd but it's unlikely to be related to Visual C++. Are the stress test clients running on the same machine?

    Since the error seems to indicate that virtual memory is a problem you should use tools like Process Explorer and VMMap (http://technet.microsoft.com/en-us/sysinternals/bb795533) to check various memory related numbers. After all it's possible that thread creation fails because something else use all the available virtual memory.

    Thursday, November 1, 2012 9:42 AM
    Moderator
  • Yes, the stress test clients are running on the same machine.

    The unusable memory increase very rapidly in the test machine, for most of the 64K memory block, the 4K of the block head is used, and the other is free, it seems very strange. It sounds we use PageHeap to record the memory allocation and unallocation, but I am sure I didn't use such tool.

    I moved the testing environment to another machine, the unusable memory increased constantly, but not very fast. 

    It should  have memory leak. I will check who are responsible for those memory. and I will feedback the progress if I had some founding.

    BTW, even using the task manager or performance monitor to monitor virtual bytes used, we could not detect the memory leak.

    Thanks.



    • Edited by xinsong001 Friday, November 2, 2012 1:18 AM
    Friday, November 2, 2012 1:16 AM
  • Sorry to tell you my result early.

    After investigation, I found the large unusable memory is due to an anti-virus software I installed, the software seems monitor and tag the memory allocation of every user mode process.


    Friday, November 9, 2012 10:23 AM