none
Is SetStdHandle(STD_OUTPUT_HANDLE, ..) broken under windows 7 ??

    Question

  • I have an windows application (i.e. no console) where I am redirecting the standard output stream to a pipe. There appears to be a problem with the SetStdHandle call when I attempt to do that redirection under windows 7, using the same mechanism that works under xp and vista. The problem does not happen when the program is run from the development environment (VS2005), nor does it happen when it is run from the command prompt.  It only seems to happen when the executable is run from a shortcut or by double clicking on it. The following code exhibits the problem:

    HANDLE hSOH_first=INVALID_HANDLE_VALUE;
    HANDLE hSOH_second=INVALID_HANDLE_VALUE;
    HANDLE hChildStdoutRd, hChildStdoutWr; 
    
    hSOH_first= GetStdHandle(STD_OUTPUT_HANDLE);
    
    SECURITY_ATTRIBUTES saAttr; 
    BOOL fSuccess; 
    
    // Set the bInheritHandle flag so pipe handles are inherited. 
    saAttr.nLength = sizeof(SECURITY_ATTRIBUTES); 
    saAttr.bInheritHandle = TRUE; 
    saAttr.lpSecurityDescriptor = NULL; 
    
    // The steps for redirecting child process's STDOUT: 
    // 1. Save current STDOUT, to be restored later. 
    // 2. Create anonymous pipe to be STDOUT for child process. 
    // 3. Set STDOUT of the parent process to be write handle to 
    // the pipe, so it is inherited by the child process. 
    
    // Create a pipe for the child process's STDOUT. 
    if (! CreatePipe(&hChildStdoutRd, &hChildStdoutWr, &saAttr, 0)) 
    {
    BOOL pipeCreateFailed=true;
    assert(pipeCreateFailed); 
    }
    
    // Set a write handle to the pipe to be STDOUT. 
    if (! SetStdHandle(STD_OUTPUT_HANDLE, hChildStdoutWr)) 
    {
    BOOL handleSettingFailed=true;
    assert(handleSettingFailed); 
    }
    
    //read it back
    hSOH_second= GetStdHandle(STD_OUTPUT_HANDLE);
    
    //check that it was set correctly
    assert(hChildStdoutWr==hSOH_second); //THIS ASSERT IS FAILING
    
    

    The final assert is failing.  It seems really strange that a SetStdHandle which is successful followed by a GetStdHandle doesn't return the same value. I have also noted that the STD_INPUT_HANDLE, and STD_ERROR_HANDLE Gets and Sets do seem to function correctly in all cases, the problem is limited to the STD_OUTPUT_HANDLE.

    Does anyone have any insights into what might be happening or is this a bug in the SetStdHandle call? I have a sample app that exhibits the problem that I can make available.

    Thanks,

    Jay

     

     

     

    • Edited by JStav Wednesday, August 11, 2010 2:54 PM reformat html
    Wednesday, August 11, 2010 2:47 PM

Answers

  • I think I may have found my problem:

    http://support.microsoft.com/kb/105305/en-us

    My application is attempting to use the Set/GetStdHandle calls before the console initiation routines are called, so it looks like I get undefined behaviour. If I add a call to AllocConsole() before this code the SetStdHandle does behave as expected in all cases.

    Now I just need to figure out how to create a console without it showing up on the screen...

    Jay

     

    Thursday, August 12, 2010 2:44 PM
  • Hi JStav,

    >>Now I just need to figure out how to create a console without it showing up on the screen...

    You can use the following statement to hide the console during the running time:

    ShowWindow( GetConsoleWindow(), SW_HIDE );
    
    

    And use this one to recover:

    ShowWindow( GetConsoleWindow(), SW_RESTORE );
    

    If you have any concern, please let me know.

    Cheers,

    Yi


    Please remember to mark the replies as answers if they help and unmark them if they provide no help.
    Welcome to the All-In-One Code Framework! If you have any feedback, please tell us.
    Friday, August 13, 2010 8:57 AM
    Moderator

All replies

  • Well, the first thing to do is check what is causing the assert to fail - is one of them an invalid handle? 

    If (as seems most likely) the reason is hSOH_second is INVALID_HANDLE_VALUE, there ought to be an error value available from GetLastError().


    Answering policy: see profile.
    Wednesday, August 11, 2010 3:32 PM
  • Thanks for you response, as it turns out neither values are INVALID_HANDLE_VALUE or 0xffffffff (my real application tests that, the code above is just a problem demo).

    At the end of that run, the values are as follows hSOH_first is 0x1aeef8, hSOH_second is 0x1aef04, and hChildStdoutWr is 0x234. It is interesting that the two GetStdHandle calls actually return different values.

    And when that code is run under the debugger the SetStdHandle call works and the values are hSOH_first is 0x2beca8, hSOH_second is 0x1e0, and hChildStdoutWr is 0x1e0.

    Jay

    • Edited by JStav Wednesday, August 11, 2010 9:06 PM added working values
    Wednesday, August 11, 2010 8:54 PM
  • I think I may have found my problem:

    http://support.microsoft.com/kb/105305/en-us

    My application is attempting to use the Set/GetStdHandle calls before the console initiation routines are called, so it looks like I get undefined behaviour. If I add a call to AllocConsole() before this code the SetStdHandle does behave as expected in all cases.

    Now I just need to figure out how to create a console without it showing up on the screen...

    Jay

     

    Thursday, August 12, 2010 2:44 PM
  • Hi JStav,

    >>Now I just need to figure out how to create a console without it showing up on the screen...

    You can use the following statement to hide the console during the running time:

    ShowWindow( GetConsoleWindow(), SW_HIDE );
    
    

    And use this one to recover:

    ShowWindow( GetConsoleWindow(), SW_RESTORE );
    

    If you have any concern, please let me know.

    Cheers,

    Yi


    Please remember to mark the replies as answers if they help and unmark them if they provide no help.
    Welcome to the All-In-One Code Framework! If you have any feedback, please tell us.
    Friday, August 13, 2010 8:57 AM
    Moderator