locked
WaitForSingleObject() method hangs if the process returns value more than 4095 bytes

    Question

  • Hi
    I tried to start a process and read the ouput of the process in c++ using the following code
      
                   BSTR Folderpath, ///< Base folder where executable is located
                   BSTR Command,  ///< Name of the command to start (notepad.exe)
                   BSTR *Param,  ///< The value to be passed to the exe
                   CHandle hStdOut;
                   CString sOutput;
                   HANDLE hProcess;
                  // Parse the parameters
                   CCallParams params(Folderpath, Command, *Param);

                  //  object to capture Standard-Output and contains STARTUP_INFO and PROCESS_INFO for the process
                      ProcessinfoStdout ioInformation;

                  // Start the process
                  hProcess = StartProcess(params, ioInformation);

                   hStdOut.Attach(ioInformation.GetStdOutHandle() );
     
                   // Wait for it to finish
                   WaitForSingleObject( hProcess, INFINITE )
                   
                    // Read its output
                    sOutput= ReadtheOutput( hStdOut, false);

    If the process(hProcess) returns value greater than 4095 bytes, the application hangs in WaitForSingleObject() line but this works fine if the return value is less than 4095 bytes. Please let me know how to handle this

    Wednesday, October 05, 2011 12:17 PM

Answers

  • Prahalnathan wrote:

    // Wait for it to finish
    WaitForSingleObject( hProcess, INFINITE )

    // Read its output
    sOutput= ReadtheOutput( hStdOut, false);

    You have to do these two tasks the other way round. The process is  sitting in WriteFile with the buffer full, waiting for you to read some  data and free up buffer space. While you are waiting for the process to  terminate.


    Igor Tandetnik

    • Marked as answer by Prahalnathan Monday, October 24, 2011 10:25 AM
    Wednesday, October 05, 2011 1:00 PM
  • Prahalnathan wrote:

    Is there i removed the waitforsingleobject(), now my doubt is there  any process whose pipes are closed but still the process runs

    Yes, it's theoretically possible. The process would have to do it very  deliberately though, it's unlikely to happen by accident.

    If that's a concern, why have you removed WaitForSingleObject call?


    Igor Tandetnik

    • Marked as answer by Prahalnathan Monday, October 24, 2011 10:24 AM
    Friday, October 14, 2011 11:53 AM

All replies

  • Prahalnathan wrote:

    // Wait for it to finish
    WaitForSingleObject( hProcess, INFINITE )

    // Read its output
    sOutput= ReadtheOutput( hStdOut, false);

    You have to do these two tasks the other way round. The process is  sitting in WriteFile with the buffer full, waiting for you to read some  data and free up buffer space. While you are waiting for the process to  terminate.


    Igor Tandetnik

    • Marked as answer by Prahalnathan Monday, October 24, 2011 10:25 AM
    Wednesday, October 05, 2011 1:00 PM
  • Hi,

    I changed the logic to read the output first and then to wait for the process to complete.This works fine in application but while dubbuging, it hangs in the WaitForSingleObject( hProcess, INFINITE ) line.

    Friday, October 07, 2011 9:34 AM
  • Prahalnathan wrote:

    I changed the logic to read the output first and then to wait for the  process to complete.This works fine in application but
    while dubbuging, it hangs in the WaitForSingleObject( hProcess,  INFINITE ) line.

    So the process doesn't terminate. Figure out why.


    Igor Tandetnik

    Friday, October 07, 2011 12:30 PM
  • Hi

    I dont think my code change will handle a situation if the process takes an hour to give the output.I do need to loop the coding to read the output until the process terminates.Can you please help to handle that

    Thanks

    Friday, October 07, 2011 12:40 PM
  • Prahalnathan wrote:

    I dont think my code change will handle a situation if the process  takes an hour to give the output.I do need to loop the coding
    to read the output until the process terminates.Can you please help to  handle that

    Which part do you have a problem with? Reading, writing a loop,  something else?


    Igor Tandetnik

    Friday, October 07, 2011 12:45 PM
  •  
                    // Read its output
                    sOutput= ReadtheOutput( hStdOut, false);             

                  // Wait for it to finish
                   WaitForSingleObject( hProcess, INFINITE )
      

    I do have problem to read the output from the filled buffer.I want a logic which reads all the output from hprocess until it terminates.

    Thanks in advance

       

    Friday, October 07, 2011 1:48 PM
  • On 10/7/2011 9:48 AM, Prahalnathan wrote:

    I do have problem to read the output from the filled buffer.I want a logic which reads all the output from hprocess until it terminates.

    Just keep calling ReadFile until it returns FALSE (an error occurred) or reports zero bytes (end-of-file reached, the other process closed its end of the pipe).

    It's not quite clear from your code how you start the child process and set up the pipes. Most likely, you use CreatePipe to obtain the two pipe handles (one for reading, one for writing). In this case, right after passing the write handle to the child process, call CloseHandle on it; otherwise ReadFile will not be able to recognize end-of-file condition.


    Igor Tandetnik

    Friday, October 07, 2011 6:21 PM
  • Hi ,

    The following code reads the value from the child process using its stdoutout handle and append the value to stingstream. will this ReadFile method reads the output from the child process till the process gets terminated?. That is if the child process gives a string as output this ReadFile method reads the output properly but if the child process gives another line as ouput after an hour will this readfile method read that. If not please tell me how to handle that

    wstringstream outstream;

    ToReadbytes = 1;

     while(ToReadbytes )

    {

     if( ::ReadFile( hStdOut, pszBuffer, bufferSize, &ToReadbytes , NULL ))

    if (dwNumRead)

    {

    _ASSERTE(ToReadbytes < sizeof(pszBuffer) );  

    memset(pszBuffer+ToReadbytes , 0,sizeof(TCHAR) );

    outstream<< pszBuffer;

    else {}; //end of data - will exit loop

    else ToReadbytes = 0; //read failed

    }

    WaitForSingleObject( hProcess, INFINITE)

     

     

    Monday, October 10, 2011 7:40 AM
  • Prahalnathan wrote:

    The following code reads the value from the child process using its  stdoutout handle

    What do you mean, using its stdoutout handle? Can you show exactly how  you call CreateProcess - in particular, how you set up STARTUPINFO  structure?

    and append the value to stingstream. will
    this ReadFile method reads the output from the child process till the  process gets terminated?. That is if the child process
    gives a string as output this ReadFile method reads the output  properly but if the child process gives another line as ouput
    after an hour will this readfile method read that.

    Note that, after writing the first string, the child process may need to  flush the stream (with FlushFileBuffers or fflush or similar).  Otherwise, the data might sit in CRT buffers for the next hour until  more data arrives, fills the buffer and causes CRT to write to the  actual underlying HANDLE.

    wstringstream outstream;

    ToReadbytes = 1;
    while(ToReadbytes )
    {
    if( ::ReadFile( hStdOut, pszBuffer, bufferSize, &ToReadbytes , NULL ))
    {
    if (dwNumRead)

    What's dwNumRead? I don't see it set anywhere.

    {
    _ASSERTE(ToReadbytes < sizeof(pszBuffer) );
    memset(pszBuffer+ToReadbytes , 0,sizeof(TCHAR) );

    Careful - the third parameter to ReadFile is the size of the buffer in  bytes, not in TCHARs. When building Unicode build, TCHAR is two bytes  large. Make sure to calculate bufferSize accordingly.

    Similarly, ToReadbytes is the number of bytes, not TCHARs, read. If  pszBuffer is TCHAR* pointer, and TCHAR is two bytes large, then  pszBuffer+ToReadbytes points to a character twice as far from the start  of the buffer as the last character read.

    Actually, there's no guarantee ReadFile will always read an even number  of bytes. It could break mid-character. It's your responsibility to  splice the two chunks properly.

    Also, is there in fact extra space in pszBuffer for the terminating NUL?


    Igor Tandetnik

    Monday, October 10, 2011 2:59 PM
  • Hi

    What do you mean, using its stdoutout handle? Can you show exactly how  you call CreateProcess - in particular, how you set up STARTUPINFO  structure?

    We do have seperate class which creates the pipes and assign values for STARTUPINFO  structures.

    StartupInfo.cb = sizeof(STARTUPINFO);

    StartupInfo.dwFlags = STARTF_USESTDHANDLES;

    StartupInfo.hStdInput = m_stdInRead;//handle of the pipe to send input to the child process

    StartupInfo.hStdOutput = m_stdOutWrite;//handle of the pipe to read output from the child process;

    StartupInfo.hStdError = m_stdErrWrite;

    The hStdOut variable in my code is the handle for the standard output. 'CreateProcessAsUser' method is used to create the process.

    Note that, after writing the first string, the child process may need to  flush the stream (with FlushFileBuffers or fflush or similar).  Otherwise, the data might sit in CRT buffers for the next hour until  more data arrives, fills the buffer and causes CRT to write to the  actual underlying HANDLE.

    The second Paramenter holds the values read from the pipe, should that buffer be flushed or the buffer of the pipe. Is there any flag which will be set if the buffer of the pipe fills.

    What's dwNumRead? I don't see it set anywhere.

    Sorry. That is ‘ToReadbytes’. The byte size allocation is handled properly for Unicode characters. Now I face problem to read the values while the child process gives the second output which is after 3 mins from the first output.

     

    Tuesday, October 11, 2011 5:05 AM
  • Prahalnathan wrote:

    We do have seperate class which creates the pipes and assign values  for STARTUPINFO structures.

    StartupInfo.cb =sizeof(STARTUPINFO);
    StartupInfo.dwFlags = STARTF_USESTDHANDLES;
    StartupInfo.hStdInput = m_stdInRead;//handle of the pipe to send  input to the child process
    StartupInfo.hStdOutput = m_stdOutWrite;//handle of the pipe to read  output from the child process;
    StartupInfo.hStdError = m_stdErrWrite;

    How is m_stdOutWrite created? If with CreatePipe, then, as I said, you  need to close it (with CloseHandle) as soon as CreateProcess returns.  Otherwise, ReadFile won't be able to detect end-of-file condition (since  the write end of the pipe is still alive, someone could conceivably  still write to it, so ReadFile will just be sitting there, waiting). You  should be reading from a corresponding read handle (recall that  CreatePipe creates a pair of handles, one for each end of the pipe)

    Note that, after writing the first string, the child process may need  to flush the stream (with FlushFileBuffers or fflush or
    similar). Otherwise, the data might sit in CRT buffers for the next  hour until more data arrives, fills the buffer and causes CRT
    to write to the actual underlying HANDLE.
     

    The second Paramenter holds the values read from the pipe, should that  buffer be flushed or the buffer of the pipe. Is there any
    flag which will be set if the buffer of the pipe fills.

    The problem is not with the buffers that the operating system maintains  - ReadFile on a pipe will see any bytes written to the pipe immediately.  But, if the child process uses something like printf() to write to its  standard output, the C Runtime (CRT) may accumulate the data in its own  buffers, before calling WriteFile on the underlying HANDLE. Naturally,  the operating system has no control over that - it only becomes aware of  the data once WriteFile is called. Only the child process can disable  CRT buffering. See fflush, setvbuf.


    Igor Tandetnik

    • Marked as answer by Prahalnathan Wednesday, October 12, 2011 6:10 AM
    • Unmarked as answer by Prahalnathan Friday, October 14, 2011 9:42 AM
    Tuesday, October 11, 2011 1:20 PM
  • Hi

    Thanks a lot. The problem was because the write end of the output handle was not closed after creating the child process.Now it is working fine.

    Wednesday, October 12, 2011 4:37 AM
  • Hi,

    Is there i removed the waitforsingleobject(), now my doubt is there any process whose pipes are closed but still the process runs

    Friday, October 14, 2011 9:43 AM
  • Prahalnathan wrote:

    Is there i removed the waitforsingleobject(), now my doubt is there  any process whose pipes are closed but still the process runs

    Yes, it's theoretically possible. The process would have to do it very  deliberately though, it's unlikely to happen by accident.

    If that's a concern, why have you removed WaitForSingleObject call?


    Igor Tandetnik

    • Marked as answer by Prahalnathan Monday, October 24, 2011 10:24 AM
    Friday, October 14, 2011 11:53 AM