none
HOWTO Capture all HTTP data with WebBrowser control.

    Question

  • HOWTO Capture all HTTP GET/POST data with WebBrowser control.

    This post is a follow up to the following post, which never got an answer.

    http://social.msdn.microsoft.com/Forums/en/csharpgeneral/thread/7b471509-897d-4b4d-9361-ce19a3fd27ca

    It left off with one suggesting using .NET 4.0 and HttpListner or Fiddler, but those do not seem right. This post is also a kind of follow up to my own thread as such: http://social.msdn.microsoft.com/Forums/en-US/winforms/thread/bee78001-8eb4-406e-9cf6-9d08f90069ec There is also a great article on extending the WebBrowser control at: http://www.codeproject.com/KB/cpp/ExtendedWebBrowser.aspx

    In it, the MSDN article: http://support.microsoft.com/kb/325079 gives the means to handle the BeforeNavigate2 event. This works fine, and captures posted data and http headers that the WebBrowser control makes, but it does not reveal all things that actually get sent via the socket communication. It only captures the HTML/DOM's HTTP-Headers and HTML-Posted data. It does not capture base HTTP HEaders like the ASPSESSIONID, or other HTTP Headers that the WebBrowser has. One can see this by running something like Fiddler, at the same time the BeforeNaviagte2 is handled. The BeforeNavigate2 will only show a HTML-Form's "method" on the HTTP HEaders, but Fiddler will show things like ASPSESSIONID.

    Now, for some history. I started using the MSHTML DOM, back in 2000 via VC++. And with C#/.NET in 2002. In those days, the WebBrowser class did not exists and I had to use tons of interoperability for the .NET. The BeforeNavigate2 then, as I remember, did trap all HTTP Headers, including ASPSESSIONID. And I would capture this header, so I could launch other WebBrowser controls, to use the same ASPSESSIONID. So, something has changed. Of course, back then, I used the RES protocol for HTML and script, which is long since blocked since XP/SP2. Also since then, things like XmlHttpRequest/AJAX have come along, as well as includes for .JS files. When things like this fire off in a WebBrowser, especially the javascript includes, the WebBrowser only reports them as the URL ==>javascript: ''<== (minue the ==> <== of course). An Example is as such: <script type="text/javascript" src="MyJavaInclude.js"></script> The BeforeNavigate2 will not capture any HTTP headers, or posted data, and the URL will only be ==>javascript: ''<==.

    At any rate, the bottom line is that now the WebBrowser control, even extended, is not getting everything. Just look at the example of a javascript include to BeforeNavigate2.  Can we work with say the ExtendedWebBrowser control, and add something to capture everything? Is perhaps there something already built into WebBrowser that has the HTTP headers to-be-sent at the time of BeforeNavigate2. Any ideas?

     

     

     

    Thursday, June 23, 2011 1:32 PM

Answers

  • Hi J-S-B,

    Welcome to the MSDN Forum.

    A WebBrowser control can not get all the data from your web server. A web server such as IIS has its own security mechanism and we can not get the sensitive information.

    I think you can use NetWork Monitor to capture the network.

    Microsoft Network Monitor 3.4
    http://www.microsoft.com/download/en/details.aspx?id=4865

    You can get the API in the api folder which is at the application path and you can also get the help document in the Help folder.

    If you have any questions, please feel free to tell us.

    Best Regards


    Neddy Ren [MSFT]
    MSDN Community Support | Feedback to us
    Get or Request Code Sample from Microsoft
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    • Marked as answer by Neddy Ren Wednesday, July 06, 2011 2:50 AM
    Wednesday, June 29, 2011 7:19 AM
  • Hello J-S-B,

    Or you can have a look at csEXWB.dll ( The most complete C# Webbrowser wrapper control It's a C# (.NET 2.0) control which creates, hosts, and offers advanced customization such as dragdrop, file downloads, HTTP/S header viewing, and much more.

    csEXWB is a C# .NET 2.0 control that creates, hosts and sinks the events of the original Webbrowser control (Not .NET or any other wrapper). Advanced customization and total control over the Webbrowser control are achieved via implementation of a number of interfaces, along with the addition of many methods, properties, events and a COM library. That includes being able to view page headers.


    Helen Zhou [MSFT]
    MSDN Community Support | Feedback to us
    Get or Request Code Sample from Microsoft
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    • Marked as answer by Neddy Ren Wednesday, July 06, 2011 2:51 AM
    Wednesday, June 29, 2011 8:39 AM

All replies

  • Hello J-S-B,

    We will do some research and  keep you update.

    Thank you for your understanding.


    Helen Zhou [MSFT]
    MSDN Community Support | Feedback to us
    Get or Request Code Sample from Microsoft
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.


    Tuesday, June 28, 2011 7:53 AM
  • Have a look at the page here:

    WinForms WebBrowser + HTTP Server:
    http://www.codeproject.com/KB/miscctrl/WebBrowserHTTP.aspx


    Knowledge will change the destiny.
    Tuesday, June 28, 2011 9:55 AM
  • Hi J-S-B,

    Welcome to the MSDN Forum.

    A WebBrowser control can not get all the data from your web server. A web server such as IIS has its own security mechanism and we can not get the sensitive information.

    I think you can use NetWork Monitor to capture the network.

    Microsoft Network Monitor 3.4
    http://www.microsoft.com/download/en/details.aspx?id=4865

    You can get the API in the api folder which is at the application path and you can also get the help document in the Help folder.

    If you have any questions, please feel free to tell us.

    Best Regards


    Neddy Ren [MSFT]
    MSDN Community Support | Feedback to us
    Get or Request Code Sample from Microsoft
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    • Marked as answer by Neddy Ren Wednesday, July 06, 2011 2:50 AM
    Wednesday, June 29, 2011 7:19 AM
  • Hello J-S-B,

    Or you can have a look at csEXWB.dll ( The most complete C# Webbrowser wrapper control It's a C# (.NET 2.0) control which creates, hosts, and offers advanced customization such as dragdrop, file downloads, HTTP/S header viewing, and much more.

    csEXWB is a C# .NET 2.0 control that creates, hosts and sinks the events of the original Webbrowser control (Not .NET or any other wrapper). Advanced customization and total control over the Webbrowser control are achieved via implementation of a number of interfaces, along with the addition of many methods, properties, events and a COM library. That includes being able to view page headers.


    Helen Zhou [MSFT]
    MSDN Community Support | Feedback to us
    Get or Request Code Sample from Microsoft
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    • Marked as answer by Neddy Ren Wednesday, July 06, 2011 2:51 AM
    Wednesday, June 29, 2011 8:39 AM
  • Thanks Helen, that article was THE ANSWER. I posted on it how I got it to work in a Hello-World VS2010 app to. Thanks everyone else for the help.
    Thursday, July 28, 2011 4:30 PM
  • The browser component that you suggested seems to have some bug related to file download events. It has stopped raising the 'FileDownloadExProgress' event.

    It has been working in our application for an year or so. It stopped doing so, couple of months back on my Windows 7 machines.

    I'm facing issues with debugging the problem, since the code for download manager has not been published! Is there any insight that you could provide?

    Monday, August 22, 2011 4:41 AM
  • Have you captured the file download events in your app? There seems to be some problem with the API; the csExWebBrowser component has stopped raising 'FileDownloadExProgress' event!
    Monday, August 22, 2011 4:48 AM