none
WebBrowser DocumentCompleted doesn't fire

    Question

  • My code scrapes info from a web site, and when the info doesn't appear correct, I load a page into a WebBrowser on my form for inspection, and load a MessageBox containing buttons to permit me to accept or reject the data. Initially, when the web page loaded I got a series of "Script Error" boxes, so I've adopted a strategy I found in Microsoft vb.net documentation as follows:

     ' Hides script errors without hiding other dialog boxes.
        Private Sub SuppressScriptErrorsOnly(ByVal WebBrowser1 As WebBrowser)
            ' Ensure that ScriptErrorsSuppressed is set to false.
            WebBrowser1.ScriptErrorsSuppressed = False
            ' Handle DocumentCompleted to gain access to the Document object.
            AddHandler WebBrowser1.DocumentCompleted, _
                AddressOf browser_DocumentCompleted

        End Sub

        Private Sub browser_DocumentCompleted(ByVal sender As Object, _
        ByVal e As WebBrowserDocumentCompletedEventArgs)
            AddHandler CType(sender, WebBrowser).Document.Window.Error, _
                AddressOf Window_Error

        End Sub

        Private Sub Window_Error(ByVal sender As Object, _
            ByVal e As HtmlElementErrorEventArgs)
            ' Ignore the error and suppress the error dialog box.
            e.Handled = True

        End Sub

    which solved the problem.  Currently I am attempting to replace the MessageBox by stopping the code when I get suspect data and using buttons on my Form1 to accept or reject the data.  My problem is that now the web page does not display in WebBrowser1.  The difference appears to be that when the MessageBox displays the browser_DocumentCompleted subroutine executes, but without the MessageBox displaying the browser_DocumentCompleted subroutine does not.

    Is there any way to force DocumentCompleted to fire without displaying a MessageBox?

    Wednesday, March 22, 2017 8:54 PM

All replies

  • Hello,

    I try your code and find the DocumentCompleted Event can be fired, I don't find any MessageBox showing.

    Best Regards,

    Cherry Bu


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Thursday, March 23, 2017 7:07 AM
    Moderator
  • The DocumentCompleted will fire only when the page finishes loading which say we look at on the web side after $(document).ready() event has completed. Keeping on the web side, a scripting error can cause code to go into a infinite loop or simply break and not ever finish because of many possible reasons thus since $(document).ready() never completes neither does the web browsers DocumentCompleted event.

    If you are simply scraping information you might look at not loading a web browser control but instead look at using the library Html Agility pack which will take time to learn how to use it yet the benefits are no scripting errors as you are not attempting to render a page but instead read it. Most examples will be in C# so you will need to do conversions. For example I took the first code block from here and did a conversion (I know C# so this was done in notepad)

    C#

    HtmlNode footer = htmlDocument
    	.DocumentNode
    	.Descendants()
    	.SingleOrDefault(x => x.Id == "footer");

    VB

    Dim footer As HtmlNode = htmlDocument _ 
    	.DocumentNode _ 
    	.Descendants() _ 
    	.SingleOrDefault(Function(x) x.Id = "footer")
    Also one going this way will need to work with say Chrome browser's Developer tools to know the elements to target with Agility pack.


    Please remember to mark the replies as answers if they help and unmark them if they provide no help, this will help others who are looking for solutions to the same or similar problem. Contact via my Twitter (Karen Payne) or Facebook (Karen Payne) via my MSDN profile but will not answer coding question on either.
    VB Forums - moderator
    profile for Karen Payne on Stack Exchange, a network of free, community-driven Q&A sites

    Thursday, March 23, 2017 10:00 AM
    Moderator
  • Your code shows the problem that the Method box in the code view of VB since 2015 is for whatever reason removed. 

    There was asked in the suggestions to add it for C# as well but they did it more simple. They removed it for VB. 

    However, the problem is now to find the arguments if you have them you can type after a method currently "Handles" then what can be used is shown. 

    Your code would be like this. 

        Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
            WebBrowser1.ScriptErrorsSuppressed = False
        End Sub
        Private Sub browser_DocumentCompleted(sender As Object, e As WebBrowserDocumentCompletedEventArgs) Handles WebBrowser1.DocumentCompleted
            AddHandler CType(sender, WebBrowser).Document.Window.Error, AddressOf Window_Error
        End Sub

    Be aware that a Page can contain more documents (frames) this can be seen if the webbrowser IsBusy becomes false. 


    Success
    Cor



    Thursday, March 23, 2017 10:49 AM
  • To reiterate what I am attempting to accomplish, and perhaps provide a bit more detail; I am scraping the SEC's Edgar website to obtain the dates on which companies I'm tracking as potential investments have reported earnings.  A typical web page from which I extract info would be:

    "https://www.sec.gov/cgi-bin/browse-edgar?action=getcompany&CIK=0000353184&type=&dateb=&owner=exclude&start=100&count=100"

    I use HttpWebRequest and WebReponse to capture the page as a text stream, and then parse the stream using a search string and Instr to find the desired info.  The problem is that there are significant differences between different companies or different time periods so that is not possible to find a search string that works reliably in all circumstances. Therefore it is frequently necessary to manually inspect the web page, and click links on it, to accept or reject the data indicated by the search string.  Currently I have been doing this by navigating manually (independent of my program) to the relevant web page using Edge, a time consuming process.

    I'd like my program to open the appropriate page for me automatically, leading to the problem I've posted.  That is, the web page does not initially display in the WebBrowser on my form, apparently since the DocumentCompleted event does not fire for the reason you give.  However, when the MsgBox displays, asking me to accept or reject the data, the DocumentCompleted event fires and the web page displays. However, in order to decide to accept or reject, I have to click a link on the loaded web page, but clicking one of the buttons in the MsgBox is the only action the program will accept. I don't understand why loading the MsgBox causes the DocumentCompleted event to fire, but the fact that it does led me to hope that there might be a work around to force it to fire and still allow me to click links in the loaded page.

    I'll look at the Html Agility pack to see if it might help.


    Friday, March 31, 2017 8:07 PM
  • SomeOtherGuy,

    As an alternate, at the website that you show:

    https://www.sec.gov/cgi-bin/browse-edgar?action=getcompany&CIK=0000353184&type=&dateb=&owner=exclude&start=100&count=100

    You'll see that there's a link for an RSS feed. It would take a while but depending on what you want, you might just spend a while to construct one or more classes and use LINQ-To-XML to parse it:

    Something to consider. :)


    "A problem well stated is a problem half solved.” - Charles F. Kettering

    Friday, March 31, 2017 9:03 PM
  • I think I may have stumbled on a horrible kluge that may help me out.  After the MsgBox instruction line

    Dim response = MsgBox(msg, 3, "Accept Report Date?")

    I've inserted a Wait() instruction, which calls a subroutine that I found on the web years ago, which pauses execution for the number of milliseconds that you pass to the sub.  If I then dismiss the MsgBox by clicking the X in the top right corner, the web page remains visible in the form's WebBrowser, and will let me navigate the displayed page until the number of milliseconds passed to the Wait() subroutine expire.  Basically I've kicked the can down the road from being able to display and navigate the appropriate web page to being able to pause execution of my program until I enter input, as opposed to waiting for a preset time.  I've found posts that give methods for this, which I've yet to attempt to implement.

    Sunday, April 2, 2017 7:07 PM