FAQ: How do I automate a web page (e.g. retrieve page text, login website, search, click button or hyperlink) via WebBrowser?

Question

How do I automate a web page (e.g. retrieve page text, login website, search, click button or hyperlink) via WebBrowser?

---

Please remember to mark the replies as answers if they help and unmark them if they provide no help.
Welcome to the All-In-One Code Framework! If you have any feedback, please tell us.

Answer 1

How do I automate a web page (e.g. retrieve page text, login website, search, click button or hyperlink) via WebBrowser?

1) Basic principle:

First load a web page on a WebBrowser object, then use the GetElementsByTagName or GetElementsByID function to locate the webpage elements in WebBrowser.Document and automate them (e.g. retrieve page text, login website, search, click button or hyperlink).

 

Beforehand, you need to find out the web elements’ source html code via view source (Right-click on web page -> "View Source" menu item).

 

For example: How do I automatically login to website (input uername/password and click Login button)?

 

Assume that this page http://www.website.com/login.aspx has the following elements:

<input name="UserNameTextBox" type="text" value="myUser" id="UserNameTextBox">

<input name="PasswordTextBox" type="text" value="myUser" id="PasswordTextBox">

<INPUT type=submit value="Login" name="LoginButton">

Public Class Form1
    Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
        ' Part 1: Use WebBrowser control to load web page
        WebBrowser1.Navigate("http://www.website.com/login.aspx")

        System.Threading.Thread.Sleep(2000) ' Delay 2 seconds to render login page

        ' Part 2: Automatically input username and password
        Dim theElementCollection As HtmlElementCollection
        theElementCollection = WebBrowser1.Document.GetElementsByTagName("input")
        For Each curElement As HtmlElement In theElementCollection
            Dim controlName As String = curElement.GetAttribute("name").ToString
            If controlName = "UserNameTextBox" Then
                curElement.SetAttribute("Value", "Username text here")
            ElseIf controlName = "PasswordTextBox" Then
                curElement.SetAttribute("Value", "Password text here")
                'In addition,you can get element value like this:
                'MessageBox.Show(curElement.GetAttribute("Value"))
            End If
        Next

        ' Part 3: Automatically clck that Login button
        theElementCollection = WebBrowser1.Document.GetElementsByTagName("input")
        For Each curElement As HtmlElement In theElementCollection
            If curElement.GetAttribute("value").Equals("Login") Then
                curElement.InvokeMember("click")
                ' javascript has a click method for you need to invoke on button and hyperlink elements.
            End If
        Next
    End Sub
End Class

 

2) Some code samples about WebBrowser Automation:

Automatically login in to the Yahoo website (Input uername/password and click Login button)

http://social.msdn.microsoft.com/Forums/en/vbgeneral/thread/0b77ca8c-48ce-4fa8-9367-c7491aa359b0/

 

Automatically execute a Windows Live search or Google (Enter search keyword and click Search button)

http://social.msdn.microsoft.com/forums/en-US/vbgeneral/thread/f06fcdca-ed9b-464b-ba08-edd26b0dc801/

 

Locate html elements: DropDown list / Radio Button and RichTextBox

http://forums.msdn.microsoft.com/en-US/vbgeneral/thread/4ec55500-56a7-4656-b4dd-1e5871fc9806/

 

Locate Hyperlink elements and retrieve URL values from the "href" attribute, then use My.Computer.Network.DownloadFile method to download the file.

http://forums.msdn.microsoft.com/en-US/vbgeneral/thread/6cd60d72-0695-4408-bf40-d858aefc8945/

 

3) Note: As for table data (without enough attributes to locate them) on a web page, you need to use Regular Expressions (System.Text.RegularExpressions namespace).

 

Related thread:

http://social.msdn.microsoft.com/forums/en-US/Vsexpressvb/thread/4795ab09-c963-4f5d-8f67-0e4717f6faed/

 

Regular Expressions Forum is here:

http://social.msdn.microsoft.com/forums/en-US/regexp/threads/

 

For more FAQ about Visual Basic .NET General, please see Visual Basic .NET General FAQ

 

---

Please remember to mark the replies as answers if they help and unmark them if they provide no help.
Welcome to the All-In-One Code Framework! If you have any feedback, please tell us.