WaitForSingleObject() method hangs if the process returns value more than 4095 bytes

Question

Hi
I tried to start a process and read the ouput of the process in c++ using the following code
  
               BSTR Folderpath, ///< Base folder where executable is located
               BSTR Command,  ///< Name of the command to start (notepad.exe)
               BSTR *Param,  ///< The value to be passed to the exe
               CHandle hStdOut;
               CString sOutput;
               HANDLE hProcess;
              // Parse the parameters
               CCallParams params(Folderpath, Command, *Param);

              //  object to capture Standard-Output and contains STARTUP_INFO and PROCESS_INFO for the process
                  ProcessinfoStdout ioInformation;

              // Start the process
              hProcess = StartProcess(params, ioInformation);

               hStdOut.Attach(ioInformation.GetStdOutHandle() );
 
               // Wait for it to finish
               WaitForSingleObject( hProcess, INFINITE )
               
                // Read its output
                sOutput= ReadtheOutput( hStdOut, false);

If the process(hProcess) returns value greater than 4095 bytes, the application hangs in WaitForSingleObject() line but this works fine if the return value is less than 4095 bytes. Please let me know how to handle this

Answer 1

Prahalnathan wrote:

// Wait for it to finish
WaitForSingleObject( hProcess, INFINITE )

// Read its output
sOutput= ReadtheOutput( hStdOut, false);

You have to do these two tasks the other way round. The process is  sitting in WriteFile with the buffer full, waiting for you to read some  data and free up buffer space. While you are waiting for the process to  terminate.

---

Igor Tandetnik

Answer 2

Hi,

I changed the logic to read the output first and then to wait for the process to complete.This works fine in application but while dubbuging, it hangs in the WaitForSingleObject( hProcess, INFINITE ) line.

Answer 3

Prahalnathan wrote:

I changed the logic to read the output first and then to wait for the  process to complete.This works fine in application but
while dubbuging, it hangs in the WaitForSingleObject( hProcess,  INFINITE ) line.

So the process doesn't terminate. Figure out why.

---

Igor Tandetnik

Answer 4

Hi

I dont think my code change will handle a situation if the process takes an hour to give the output.I do need to loop the coding to read the output until the process terminates.Can you please help to handle that

Thanks

Answer 5

Prahalnathan wrote:

I dont think my code change will handle a situation if the process  takes an hour to give the output.I do need to loop the coding
to read the output until the process terminates.Can you please help to  handle that

Which part do you have a problem with? Reading, writing a loop,  something else?

---

Igor Tandetnik

Answer 6

 
                // Read its output
                sOutput= ReadtheOutput( hStdOut, false);             

              // Wait for it to finish
               WaitForSingleObject( hProcess, INFINITE )
  

I do have problem to read the output from the filled buffer.I want a logic which reads all the output from hprocess until it terminates.

Thanks in advance

   

Answer 7

On 10/7/2011 9:48 AM, Prahalnathan wrote:

I do have problem to read the output from the filled buffer.I want a logic which reads all the output from hprocess until it terminates.

Just keep calling ReadFile until it returns FALSE (an error occurred) or reports zero bytes (end-of-file reached, the other process closed its end of the pipe).

It's not quite clear from your code how you start the child process and set up the pipes. Most likely, you use CreatePipe to obtain the two pipe handles (one for reading, one for writing). In this case, right after passing the write handle to the child process, call CloseHandle on it; otherwise ReadFile will not be able to recognize end-of-file condition.

---

Igor Tandetnik

Answer 8

Hi ,

The following code reads the value from the child process using its stdoutout handle and append the value to stingstream. will this ReadFile method reads the output from the child process till the process gets terminated?. That is if the child process gives a string as output this ReadFile method reads the output properly but if the child process gives another line as ouput after an hour will this readfile method read that. If not please tell me how to handle that

wstringstream outstream;

ToReadbytes = 1;

 while(ToReadbytes )

{

 if( ::ReadFile( hStdOut, pszBuffer, bufferSize, &ToReadbytes , NULL ))

{ 

if (dwNumRead)

{

_ASSERTE(ToReadbytes < sizeof(pszBuffer) );  

memset(pszBuffer+ToReadbytes , 0,sizeof(TCHAR) );

outstream<< pszBuffer;

} 

else {}; //end of data - will exit loop

} 

else ToReadbytes = 0; //read failed

}

WaitForSingleObject( hProcess, INFINITE)

 

 

Answer 9

Prahalnathan wrote:

The following code reads the value from the child process using its  stdoutout handle

What do you mean, using its stdoutout handle? Can you show exactly how  you call CreateProcess - in particular, how you set up STARTUPINFO  structure?

and append the value to stingstream. will
this ReadFile method reads the output from the child process till the  process gets terminated?. That is if the child process
gives a string as output this ReadFile method reads the output  properly but if the child process gives another line as ouput
after an hour will this readfile method read that.

Note that, after writing the first string, the child process may need to  flush the stream (with FlushFileBuffers or fflush or similar).  Otherwise, the data might sit in CRT buffers for the next hour until  more data arrives, fills the buffer and causes CRT to write to the  actual underlying HANDLE.

wstringstream outstream;

ToReadbytes = 1;
while(ToReadbytes )
{
if( ::ReadFile( hStdOut, pszBuffer, bufferSize, &ToReadbytes , NULL ))
{
if (dwNumRead)

What's dwNumRead? I don't see it set anywhere.

{
_ASSERTE(ToReadbytes < sizeof(pszBuffer) );
memset(pszBuffer+ToReadbytes , 0,sizeof(TCHAR) );

Careful - the third parameter to ReadFile is the size of the buffer in  bytes, not in TCHARs. When building Unicode build, TCHAR is two bytes  large. Make sure to calculate bufferSize accordingly.

Similarly, ToReadbytes is the number of bytes, not TCHARs, read. If  pszBuffer is TCHAR* pointer, and TCHAR is two bytes large, then  pszBuffer+ToReadbytes points to a character twice as far from the start  of the buffer as the last character read.

Actually, there's no guarantee ReadFile will always read an even number  of bytes. It could break mid-character. It's your responsibility to  splice the two chunks properly.

Also, is there in fact extra space in pszBuffer for the terminating NUL?

---

Igor Tandetnik

Answer 10

Hi

What do you mean, using its stdoutout handle? Can you show exactly how  you call CreateProcess - in particular, how you set up STARTUPINFO  structure?

We do have seperate class which creates the pipes and assign values for STARTUPINFO  structures.

StartupInfo.cb = sizeof(STARTUPINFO);

StartupInfo.dwFlags = STARTF_USESTDHANDLES;

StartupInfo.hStdInput = m_stdInRead;//handle of the pipe to send input to the child process

StartupInfo.hStdOutput = m_stdOutWrite;//handle of the pipe to read output from the child process;

StartupInfo.hStdError = m_stdErrWrite;

The hStdOut variable in my code is the handle for the standard output. 'CreateProcessAsUser' method is used to create the process.

Note that, after writing the first string, the child process may need to  flush the stream (with FlushFileBuffers or fflush or similar).  Otherwise, the data might sit in CRT buffers for the next hour until  more data arrives, fills the buffer and causes CRT to write to the  actual underlying HANDLE.

The second Paramenter holds the values read from the pipe, should that buffer be flushed or the buffer of the pipe. Is there any flag which will be set if the buffer of the pipe fills.

What's dwNumRead? I don't see it set anywhere.

Sorry. That is ‘ToReadbytes’. The byte size allocation is handled properly for Unicode characters. Now I face problem to read the values while the child process gives the second output which is after 3 mins from the first output.

 

Answer 11

Prahalnathan wrote:

We do have seperate class which creates the pipes and assign values  for STARTUPINFO structures.

StartupInfo.cb =sizeof(STARTUPINFO);
StartupInfo.dwFlags = STARTF_USESTDHANDLES;
StartupInfo.hStdInput = m_stdInRead;//handle of the pipe to send  input to the child process
StartupInfo.hStdOutput = m_stdOutWrite;//handle of the pipe to read  output from the child process;
StartupInfo.hStdError = m_stdErrWrite;

How is m_stdOutWrite created? If with CreatePipe, then, as I said, you  need to close it (with CloseHandle) as soon as CreateProcess returns.  Otherwise, ReadFile won't be able to detect end-of-file condition (since  the write end of the pipe is still alive, someone could conceivably  still write to it, so ReadFile will just be sitting there, waiting). You  should be reading from a corresponding read handle (recall that  CreatePipe creates a pair of handles, one for each end of the pipe)

Note that, after writing the first string, the child process may need  to flush the stream (with FlushFileBuffers or fflush or
similar). Otherwise, the data might sit in CRT buffers for the next  hour until more data arrives, fills the buffer and causes CRT
to write to the actual underlying HANDLE. 

The second Paramenter holds the values read from the pipe, should that  buffer be flushed or the buffer of the pipe. Is there any
flag which will be set if the buffer of the pipe fills.

The problem is not with the buffers that the operating system maintains  - ReadFile on a pipe will see any bytes written to the pipe immediately.  But, if the child process uses something like printf() to write to its  standard output, the C Runtime (CRT) may accumulate the data in its own  buffers, before calling WriteFile on the underlying HANDLE. Naturally,  the operating system has no control over that - it only becomes aware of  the data once WriteFile is called. Only the child process can disable  CRT buffering. See fflush, setvbuf.

---

Igor Tandetnik

Answer 12

Hi

Thanks a lot. The problem was because the write end of the output handle was not closed after creating the child process.Now it is working fine.

Answer 13

Hi,

Is there i removed the waitforsingleobject(), now my doubt is there any process whose pipes are closed but still the process runs

Answer 14

Prahalnathan wrote:

Is there i removed the waitforsingleobject(), now my doubt is there  any process whose pipes are closed but still the process runs

Yes, it's theoretically possible. The process would have to do it very  deliberately though, it's unlikely to happen by accident.

If that's a concern, why have you removed WaitForSingleObject call?

---

Igor Tandetnik