none
C# How to catch stdout when p-invoking to unmanaged DLL, as Visual Studio already does in the output window? RRS feed

  • Question

  • Hi to all,
    as the title states, I need to capture the messages (stdout and stderr) that an unmanaged dll sends when particular functions are called, preferably through asynchronous callbacks.
    If I enable unmanaged code debugging, I can see that VS succesfully catches this messages and shows them in the output window, but after hours of researches I still have not found a way to access them programmatically.

    I have no access to the source code of the library, but if VS can do this, I don't see any reason on why my application cannot do the same. 

    Any help would be really appreciated.
    Thanks
    Wednesday, May 9, 2012 7:40 PM

Answers

  • Hello mark_555,

    1. >> I start to think that what I want to do has nothing to do with Stdout or Stderr...

    1.1 My gut feel is that the strings are output via calls to the OutputDebugString() API.

    1.2 See http://msdn.microsoft.com/en-us/library/windows/desktop/aa363362(v=vs.85).aspx for more details.

    2. To confirm that they are indeed the result of OutputDebugString(), download DebugView from SystemInternals.com.

    2.1 Then run your application on its own (not via the Visual Studio debugger).

    2.2 If the messages apprear on the DebugView window, OutputDebugString() is indeed at work here.

    2.3 Please confirm this before proceeding with my suggestions below concerning how to access these messages for yourself.

    3. If OutputDebugString() is indeed used, there are 2 ways to access the output strings :

    3.1 By using API hooking to hook OutputDebugString().

    This is not easy and, with new versions of Windows, where security gets tighter and tighter, gets harder and harder to accomplish. I recommend that you refer to an article by John Robbins which specifically talks about hooking OutputDebugString() : http://www.microsoft.com/msj/0298/bugslayer0298.aspx.

    The article is very old (1998) and the code uses Visual Studio 5.0. One of the COM servers even uses an old outdated version of the IObjectSafetyImpl ATL class which you would need to modify a little.

    However, the code does work and, with a small amount of changes, you may find it useful. I tested it on Windows XP. But I have a feeling it may not work on Windows Vista onwards.

    3.2 By writing a minimal debugger application.

    Your debugger will call CreateProcess() with the DEBUG_PROCESS flag (which starts the target application from ground up) or calls DebugActiveProcess() (which debugs a running application). Lookup MSDN for documentation on these APIs.

    Your code must provide a debugging loop where it waits for and receives debug events. One of these debug events is OUTPUT_DEBUG_STRING_EVENT. This event is triggered when the target debuggee calls the OutputDebugString() API. When you receive this event, you will receive information on the string to be output.

    See "Writing the Debugger's Main Loop" (http://msdn.microsoft.com/en-us/library/windows/desktop/ms681675(v=vs.85).aspx) for more information.

    4. I recommend the minimal debugger approach. You will essentially mimick the workings of the Visual Studio debugger and stand a chance to learn many facinating stuff behind the workings of a debugger.

    - Bio.


    Please visit my blog : http://limbioliong.wordpress.com/

    • Proposed as answer by CrazyGhost_Von Thursday, May 10, 2012 1:58 PM
    • Marked as answer by mark_555 Saturday, May 12, 2012 6:10 PM
    Thursday, May 10, 2012 5:53 AM
  • Hello mark_555,

    1. Interesting application, the DebugMonitor class. Thanks for sharing it with us.

    2. >> Is there a way to filter the messages that only belong to the current application domain directly from the API ? ...A simple "If (pid == Process.GetCurrentProcess().Id) { "Process data..." }" does the job...

    2.1 Your simple "if" statmement is already a good enough filter, in my opinion.

    2.2 The other way is to modify the DebugMonitor class itself to expose a Process ID property that it can use to determine whether the OutputDebugString() call was from a particular process.

    2.3 The OnOutputDebugStringHandler delegate would have to be modified (i.e. minus the "pid" parameter).

    2.4 The DebugMonitor.Capture() function would have to check the current process ID from the shared memory and then invoke the (modified) OnOutputDebugStringHandler delegate if appropriate.

    - Bio.


    Please visit my blog : http://limbioliong.wordpress.com/

    • Proposed as answer by Sharad Birajdar Friday, May 11, 2012 5:32 AM
    • Marked as answer by mark_555 Saturday, May 12, 2012 6:10 PM
    Friday, May 11, 2012 4:55 AM

All replies

  • I haven't done it myself,
    but I think you can use System.Console.Out and System.Console.Error
    in non-Console apps.
    Since the pInvoked dll runs in-process it shares
    the stdout/in/err of the current process.
    Console.Out / Error are TextReaders, you can read synchronously from.
    As of .NET 4.5 also async reading is available

    Note that System.Diagnostics.Process is capabale to read stdout and stderr, even asynchronously
    The downside is that you have to set some startup flags before launching
    the Process, So you'd need also a launcher app to apply these settings

    Chris




    Wednesday, May 9, 2012 8:23 PM
  • Hi Chris and thanks for your great tips. Unfortunately didn't worked. It works with "Console.WriteLine("MyMessage!"), but looks like doesn't even attempts to catch any message from the unmanaged dll:

    FileStream fs = new FileStream("Test.txt", FileMode.Create);
    TextWriter tmp = Console.Out;
    StreamWriter sw = new StreamWriter(fs);
    Console.SetOut(sw);
    MyUnmanagedFunction(); //stdout captured from VS output window, but not from "Console.SetOut()"
    Console.WriteLine("Hello file"); //stdout captured from "Console.SetOut()"
    Console.SetOut(tmp);
    sw.Close();

    Enabling/disabling unmanaged code debugging doesn't make any difference. 

    Is there something that I forgot ?

    Wednesday, May 9, 2012 8:58 PM
  • Sorry, no.
    I have no practical experience with reading
    unmanaged stdout from C#. It was just an idea.
    Maybe it's simply not supported.

    Chris

    Wednesday, May 9, 2012 9:20 PM
  • "I can see that VS succesfully catches this messages"

    What kind of messages are you talking about? You mentioned stdout and stderr but I don't remember VS ever displaying those in the output window, what it displays are debug messages and it does that because it is a debugger.

    Wednesday, May 9, 2012 9:50 PM
    Moderator
  • Well, the main dll where VS catch the messages is called ASL.dll (Apple System Log). I just realized it is open source ( asl.c , asl.h ).
    Basically there are several dlls that send their own messages to ASL which writes them in a file.
    I don't know how VS does, but seems that it shows all of this messages in the output window and I want to catch those messages in my application in order to implement some sort of custom log.
    I tried so many solutions around the web like for example "SetStdHandle", "CreateFile" and so on, but none of them worked. I start to think that what I want to do has nothing to do with Stdout or Stderr.
    When a dll send a message to ASL, the VS output window show something like:
    "[(unknown facility) MyApp.exe] FunctionName (thread 3164): returned 0x0. Succesfully received application data."

    If I disable unmanaged code debugging, these messages disappear from the output window.
    If it's possible to access these messages as soon as they are delivered, in a similar manner as VS already does, it would be great. If I have to resort to unreliable hacks like parsing the output file(where lacks most of the interesting messages), I prefer to abbandone the whole thing.

    When I saw that VS was showing those messages, I thought that it was not so hard to access them programmatically, but probably I was wrong.

    Many thanks
    Wednesday, May 9, 2012 10:59 PM
  • Hello mark_555,

    1. >> I start to think that what I want to do has nothing to do with Stdout or Stderr...

    1.1 My gut feel is that the strings are output via calls to the OutputDebugString() API.

    1.2 See http://msdn.microsoft.com/en-us/library/windows/desktop/aa363362(v=vs.85).aspx for more details.

    2. To confirm that they are indeed the result of OutputDebugString(), download DebugView from SystemInternals.com.

    2.1 Then run your application on its own (not via the Visual Studio debugger).

    2.2 If the messages apprear on the DebugView window, OutputDebugString() is indeed at work here.

    2.3 Please confirm this before proceeding with my suggestions below concerning how to access these messages for yourself.

    3. If OutputDebugString() is indeed used, there are 2 ways to access the output strings :

    3.1 By using API hooking to hook OutputDebugString().

    This is not easy and, with new versions of Windows, where security gets tighter and tighter, gets harder and harder to accomplish. I recommend that you refer to an article by John Robbins which specifically talks about hooking OutputDebugString() : http://www.microsoft.com/msj/0298/bugslayer0298.aspx.

    The article is very old (1998) and the code uses Visual Studio 5.0. One of the COM servers even uses an old outdated version of the IObjectSafetyImpl ATL class which you would need to modify a little.

    However, the code does work and, with a small amount of changes, you may find it useful. I tested it on Windows XP. But I have a feeling it may not work on Windows Vista onwards.

    3.2 By writing a minimal debugger application.

    Your debugger will call CreateProcess() with the DEBUG_PROCESS flag (which starts the target application from ground up) or calls DebugActiveProcess() (which debugs a running application). Lookup MSDN for documentation on these APIs.

    Your code must provide a debugging loop where it waits for and receives debug events. One of these debug events is OUTPUT_DEBUG_STRING_EVENT. This event is triggered when the target debuggee calls the OutputDebugString() API. When you receive this event, you will receive information on the string to be output.

    See "Writing the Debugger's Main Loop" (http://msdn.microsoft.com/en-us/library/windows/desktop/ms681675(v=vs.85).aspx) for more information.

    4. I recommend the minimal debugger approach. You will essentially mimick the workings of the Visual Studio debugger and stand a chance to learn many facinating stuff behind the workings of a debugger.

    - Bio.


    Please visit my blog : http://limbioliong.wordpress.com/

    • Proposed as answer by CrazyGhost_Von Thursday, May 10, 2012 1:58 PM
    • Marked as answer by mark_555 Saturday, May 12, 2012 6:10 PM
    Thursday, May 10, 2012 5:53 AM
  • Hi Bio,
    let me tell you that you're a genius!! Yes, the strings are output via calls to the OutputDebugString() API as also confirmed from DebugView.
    I found the following class which already does all the dirty job of hooking the messages and to deliver them via callbacks:

    DebugMonitor.Start();
    DebugMonitor.OnOutputDebugString += new OnOutputDebugStringHandler(OnOutputDebugString);
    DebugMonitor.Stop();

    I have already implemented the class and seems to work only when I execute my application on its own.

    If I run my application inside visual studio, it hangs frequently and instead of the real strings, I receive strange messages like that:
    "*** HR originated: -2147024774\n***   Source File: d:\\iso_whid\\x86fre\\base\\isolation\\com\\copyout.cpp, line 1391"

    1- Is this the normal behaviour ? I suppose I will have to disable permanently any execution of the class when I works in VS, is that right ?

    2- I noticed that my application also captures messages from other applications that calls the same dll. How I can prevent this and ensure that I will only receives messages related to my application?

    Thanks

    EDIT:

    I think I found the answers:
    1- Yes, appears to be the normal behavior: "This class captures all strings passed to OutputDebugString when the application is not debugged.".
    I used the following conditional statement to prevent the class from being executed when I'm working on VS:
    If (!Debugger.IsAttached) { DebugMonitor.Start(); }

    2-Fortunately the callbacks also returns the process id on which a particular message belongs. A simple "If (pid == Process.GetCurrentProcess().Id) { "Process data..." }" does the job.
    Is there a way to filter the messages that only belong to the current application domain directly from the API ?

    Thanks


    • Edited by mark_555 Thursday, May 10, 2012 4:50 PM Added details
    Thursday, May 10, 2012 3:29 PM
  • Hello mark_555,

    1. Interesting application, the DebugMonitor class. Thanks for sharing it with us.

    2. >> Is there a way to filter the messages that only belong to the current application domain directly from the API ? ...A simple "If (pid == Process.GetCurrentProcess().Id) { "Process data..." }" does the job...

    2.1 Your simple "if" statmement is already a good enough filter, in my opinion.

    2.2 The other way is to modify the DebugMonitor class itself to expose a Process ID property that it can use to determine whether the OutputDebugString() call was from a particular process.

    2.3 The OnOutputDebugStringHandler delegate would have to be modified (i.e. minus the "pid" parameter).

    2.4 The DebugMonitor.Capture() function would have to check the current process ID from the shared memory and then invoke the (modified) OnOutputDebugStringHandler delegate if appropriate.

    - Bio.


    Please visit my blog : http://limbioliong.wordpress.com/

    • Proposed as answer by Sharad Birajdar Friday, May 11, 2012 5:32 AM
    • Marked as answer by mark_555 Saturday, May 12, 2012 6:10 PM
    Friday, May 11, 2012 4:55 AM
  • Thanks for your help. I just realized that I only get a few messages coming from others processes and therefore a simple "if" statement will be fine.
    At this point I can close this thread, thanks to Bio which immediately understood the problem and proposed a 100% working solution.

    Thanks
    Saturday, May 12, 2012 6:10 PM
  • I had a similar need, but with a managed DLL. There was some initialization code in the DLL that was emitting an error on StdErr that was not really a problem for the app but cluttered the command line app's normal output.

    To solve this, I briefly redirected StdErr to a dummy StringWriter sink, then reset the normal StdOut stream like this:          

                StringWriter dummyStdErr = new StringWriter();
    TextWriter stdErr = System.Console.Error; System.Console.SetError(dummyStdErr); // invoke code here that is emitting the bogus messages ... // rewire the normal stderr stream System.Console.SetError(stdErr);


    The same approach could be used for StdOut if you use SetOut/OpenStandardOut. You could also theoretically filter the output and pass any output you deem worthy along if you created a wrapper for the default StdErr writer and assigned that stream using SetError like above. I didn't need the output, so I simply discarded it.


    • Proposed as answer by Kirtlander Tuesday, July 23, 2013 1:54 PM
    • Edited by Kirtlander Tuesday, July 23, 2013 3:07 PM bug in code
    Tuesday, July 23, 2013 1:54 PM