locked
DebugObject Handle Leak on Spawning Child Processes RRS feed

  • Question

  • I have developed a Windows Service application that needs to use the functionality of third-party libraries.

    Unfortunately these libraries have memory leaks, and sometimes they even crash the whole process. Therefore I've decided to call-up this functionality from a separate process that is spawned by the Windows Service.

    To have better control on what's going (e.g. to find out which type of exception occurred in the third-party libraries) the service also debugs the spawned process (by using the DEBUG_ONLY_THIS_PROCESS flag in the call of CreateProcess). As soon as the child process exceeds a predefined memory threshold, the service ends the child process (by sending a "quit"-command over a pipe) and starts a new child process. If the child process crashes, the service starts a new child process as well.

    After the service has been running for a week (on Windows Server 2008 and Windows 7), I noticed in the Task Manager that the handle count of the Windows Service had reached several thousands was steadily increasing. Using the Process Explorer tool I found out that there were thousands of DebugObject handles. Each time the Windows Service spawned a new child process, one DebugObject handle was added and was not released after the child process exited.

    I first supposed some leak in my code, but could not find one. Then I tried other debuggers (built-in debugger of Visual Studio 2008 and Visual Studio 2010, WinDbg 6.2.9200.16384) to debug the following minimal debugee process:

    int main()
    {
      return 0;
    }

    Each time I executed this process from Visual Studio or WinDbg, a DebugObject was created and not closed. Therefore I suppose that my code is ok, because the same happens in other debuggers. The behavior can be reproduced by the following minimal debugger:

    #include <windows.h>
    #include <stdio.h>
    #include <conio.h>
    
    void DisplayLastError();
    
    int wmain()
    {
      wprintf(L"Press key to start debuggee\n");
      _getwch();
      
      WCHAR szCommandLine[256];
      wcscpy_s(szCommandLine, 256, L"debuggee.exe");
    
      DWORD dwFlags = CREATE_NO_WINDOW | CREATE_UNICODE_ENVIRONMENT | DEBUG_ONLY_THIS_PROCESS;
    
      STARTUPINFOW sui;
      sui.cb = sizeof(STARTUPINFOW);
      sui.lpReserved = NULL;
      sui.lpDesktop = NULL;
      sui.lpTitle = NULL;
      sui.dwFlags = 0;
      sui.cbReserved2 = 0;
      sui.lpReserved2 = NULL;
    
      PROCESS_INFORMATION pi;
      if (!CreateProcessW(NULL, szCommandLine, NULL, NULL, FALSE, dwFlags,
                          NULL, NULL, &sui, &pi))
      {
        DisplayLastError();
        _getwch();
        exit(-1);
      }
      CloseHandle(pi.hProcess);
      CloseHandle(pi.hThread);
    
      DEBUG_EVENT de;
      BOOL cont = TRUE;
      while (cont)
      {
        if (!WaitForDebugEvent(&de, INFINITE))
        {
          DisplayLastError();
          break;
        }
        switch (de.dwDebugEventCode)
        {
        case CREATE_PROCESS_DEBUG_EVENT:
          CloseHandle(de.u.CreateProcessInfo.hFile);
          ContinueDebugEvent(de.dwProcessId, de.dwThreadId, DBG_EXCEPTION_NOT_HANDLED);
          break;
    
        case LOAD_DLL_DEBUG_EVENT:
          CloseHandle(de.u.LoadDll.hFile);
          ContinueDebugEvent(de.dwProcessId, de.dwThreadId, DBG_EXCEPTION_NOT_HANDLED);
          break;
    
        case EXCEPTION_DEBUG_EVENT:
          if (de.u.Exception.ExceptionRecord.ExceptionCode == EXCEPTION_BREAKPOINT)
            ContinueDebugEvent(de.dwProcessId, de.dwThreadId, DBG_CONTINUE);
          else
            ContinueDebugEvent(de.dwProcessId, de.dwThreadId, DBG_EXCEPTION_NOT_HANDLED);
          break;
    
        case EXIT_PROCESS_DEBUG_EVENT:
          ContinueDebugEvent(de.dwProcessId, de.dwThreadId, DBG_EXCEPTION_NOT_HANDLED);
          cont = FALSE;
          break;
    
        default:
          ContinueDebugEvent(de.dwProcessId, de.dwThreadId, DBG_EXCEPTION_NOT_HANDLED);
          break;
        }
      }
    
      wprintf(L"Finished\n");
      _getwch();
    
      return 0;
    }
    
    void DisplayLastError()
    {
      DWORD dwErrorCode = GetLastError();
      LPVOID lpMsgBuf;
      FormatMessageW(
        FORMAT_MESSAGE_ALLOCATE_BUFFER |
        FORMAT_MESSAGE_FROM_SYSTEM |
        FORMAT_MESSAGE_IGNORE_INSERTS,
        NULL,
        dwErrorCode,
        MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT),
        (LPWSTR) &lpMsgBuf,
        0, NULL);
      wprintf(L"%s", (LPWSTR)lpMsgBuf);
      LocalFree(lpMsgBuf);
    }

    When debugging the debugger with WinDbg, !htrace reveals that the debug object is created during the call of CreateProcess:

    0x775c069e: ntdll!ZwCreateDebugObject+0x00000012
    0x7763f7b6: ntdll!DbgUiConnectToDbg+0x0000004b
    0x7691f0c9: kernel32!CreateProcessInternalW+0x00000300
    0x768e1069: kernel32!CreateProcessW+0x0000002c

    But when the debuggee quits, the object is not released.

    I've found out that I can retrieve the debug object handle by calling DbgUiGetThreadDebugObject from ntdll.dll. Then I can call CloseHandle and the debug object is gone. But I don't think that's the way it is supposed to be done.

    Is this a bug in the user-mode debugger or did I just miss something?

    Thanks for your answers and regards,
    Stefan


    • Edited by stefan.uhrig Wednesday, March 13, 2013 8:35 PM Added platforms the issue appears
    Wednesday, March 13, 2013 8:34 PM

All replies

  • FWIW, I had the same problem. Thanks for pointing me in the right direction to solve it, Stefan!!

    My debugger worked a little bit different than yours (I used CreateProcess without DEBUG-flag, but with SUSPENDED flag, and then attached a debugger using DebugActiveProcess). My guess is that it's a bug in the DebugActiveProcessStop function...

    Friday, January 24, 2014 11:06 AM