locked
FAQ: How do I automate a web page (e.g. retrieve page text, login website, search, click button or hyperlink) via WebBrowser?

    Question

  • How do I automate a web page (e.g. retrieve page text, login website, search, click button or hyperlink) via WebBrowser?


    Please remember to mark the replies as answers if they help and unmark them if they provide no help.
    Welcome to the All-In-One Code Framework! If you have any feedback, please tell us.
    Saturday, April 11, 2009 11:12 AM

Answers

  • How do I automate a web page (e.g. retrieve page text, login website, search, click button or hyperlink) via WebBrowser?


    1) Basic principle:


    First load a web page on a WebBrowser object, then use the GetElementsByTagName or GetElementsByID function to locate the webpage elements in WebBrowser.Document and automate them (e.g. retrieve page text, login website, search, click button or hyperlink).

     

    Beforehand, you need to find out the web elements’ source html code via view source (Right-click on web page -> "View Source" menu item).

     

    For example: How do I automatically login to website (input uername/password and click Login button)?

     

    Assume that this page http://www.website.com/login.aspx has the following elements:

    <input name="UserNameTextBox" type="text" value="myUser" id="UserNameTextBox">

    <input name="PasswordTextBox" type="text" value="myUser" id="PasswordTextBox">

    <INPUT type=submit value="Login" name="LoginButton">


    Public Class Form1
        Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
            ' Part 1: Use WebBrowser control to load web page
            WebBrowser1.Navigate("http://www.website.com/login.aspx")
    
            System.Threading.Thread.Sleep(2000) ' Delay 2 seconds to render login page
    
            ' Part 2: Automatically input username and password
            Dim theElementCollection As HtmlElementCollection
            theElementCollection = WebBrowser1.Document.GetElementsByTagName("input")
            For Each curElement As HtmlElement In theElementCollection
                Dim controlName As String = curElement.GetAttribute("name").ToString
                If controlName = "UserNameTextBox" Then
                    curElement.SetAttribute("Value", "Username text here")
                ElseIf controlName = "PasswordTextBox" Then
                    curElement.SetAttribute("Value", "Password text here")
                    'In addition,you can get element value like this:
                    'MessageBox.Show(curElement.GetAttribute("Value"))
                End If
            Next
    
            ' Part 3: Automatically clck that Login button
            theElementCollection = WebBrowser1.Document.GetElementsByTagName("input")
            For Each curElement As HtmlElement In theElementCollection
                If curElement.GetAttribute("value").Equals("Login") Then
                    curElement.InvokeMember("click")
                    ' javascript has a click method for you need to invoke on button and hyperlink elements.
                End If
            Next
        End Sub
    End Class
    

     

    2) Some code samples about WebBrowser Automation:


    Automatically login in to the Yahoo website (Input uername/password and click Login button)

    http://social.msdn.microsoft.com/Forums/en/vbgeneral/thread/0b77ca8c-48ce-4fa8-9367-c7491aa359b0/

     

    Automatically execute a Windows Live search or Google (Enter search keyword and click Search button)

    http://social.msdn.microsoft.com/forums/en-US/vbgeneral/thread/f06fcdca-ed9b-464b-ba08-edd26b0dc801/

     

    Locate html elements: DropDown list / Radio Button and RichTextBox

    http://forums.msdn.microsoft.com/en-US/vbgeneral/thread/4ec55500-56a7-4656-b4dd-1e5871fc9806/

     

    Locate Hyperlink elements and retrieve URL values from the "href" attribute, then use My.Computer.Network.DownloadFile method to download the file.

    http://forums.msdn.microsoft.com/en-US/vbgeneral/thread/6cd60d72-0695-4408-bf40-d858aefc8945/

     

    3) Note: As for table data (without enough attributes to locate them) on a web page, you need to use Regular Expressions (System.Text.RegularExpressions namespace).

     

    Related thread:

    http://social.msdn.microsoft.com/forums/en-US/Vsexpressvb/thread/4795ab09-c963-4f5d-8f67-0e4717f6faed/

     

    Regular Expressions Forum is here:

    http://social.msdn.microsoft.com/forums/en-US/regexp/threads/

     

    For more FAQ about Visual Basic .NET General, please see Visual Basic .NET General FAQ

     


    Please remember to mark the replies as answers if they help and unmark them if they provide no help.
    Welcome to the All-In-One Code Framework! If you have any feedback, please tell us.
    Saturday, April 11, 2009 11:14 AM