none
Creating a functional interface to remote website

    Question

  • Context:  Visual Basic.Net, operating with Visual Studio 2017 IDE, Operating in Win 7 Pro environment.

    Project objective:  Open a remote URL and display the webpage.  Push data into two input text boxes (Address 1 and Address 2) and then remotely click the 'Submit' button associated with the input boxes.  The functionality is to be achieved with Visual Basic code and not by Visual click and drag methods of Visual Studio (if at all possible)

    Problem:  After extensive searches, I've found various solutions for this task but none of the code strings are accepted by the Visual Studio Intellisense or compiler.  At my current high level of ignorance, I have two questions:

    (1) Is any sort of external 'add in' required into order to install an internet web interface, or an Internet Explorer interface, so that the Visual Basic.net IDE will recognize the typed code and compile it properly?  I would like the solution to work for any of the common browsers that a user might have installed on his computer, and not be limited to only Internet Explorer but if that is not possible, I'd take an IE solution.

    (2) What is the code necessary to open a remote website, navigate to a couple of text boxes (Address 1 and Address 2), load address information (either latitude and longitude, or city and state) and then submit the data to the website (button titled 'Submit')?

    Bob Goodwin, bobgn1@cox.net


    Bob Goodwin

    After a number of helpful suggestions, I have made a number of improvements in the available references and seem to have resolved the problem where VS 2017 VB.net does not accept the commands.  At this point, the problem seems to be getting the right command syntax to do the job.  I am trying to open a remove website (this works) and then search for an input box named 'Address1" (labeled Point A) and "Address2" (Labeled Point B) on the opened webpage and load the values for Address1 and Address2 with string variables.  At this point, the code needs to submit the Address data by 'clicking' the Submit button of the webpage as soon as Address2 is loaded. 
    The code below opens the remote website but gives me an error [Object reference not set to an instance of an object] at the For Each line.  I have been trying to get the "GetElementsByName" method to work but have not been able to make that work either.

    Can anyone whip this code segment into shape?

    Private Sub OptContactsPlot_Click(sender As Object, e As EventArgs) Handles OptContactsPlot.Click
            Dim varCityState As String = "Newport News VA"
            Dim varLatLong As String = "37.05, -76.49"
            Dim objIE As SHDocVw.InternetExplorer
            Dim htmlDoc As mshtml.HTMLDocument
            Dim htmlInput As Object
            Dim url As String
            url = "www.ecars7255.com/FrontPageLinks/NetManagement/Mapping/MapContacts.html"
            objIE = New SHDocVw.InternetExplorer
            objIE.Navigate(url)
            objIE.Visible = True
            With objIE
                Dim htmlColl As mshtml.HTMLElementCollection
                For Each htmlInput In htmlColl
                    If htmlInput.name = "Address1" Then
                        htmlInput.value = varLatLong
                    End If
                    If htmlInput.name = "Address2" Then
                        htmlInput.value = varCityState
                    End If
                Next htmlInput
    
            End With
    
        End Sub
    Bob Goodwin


    • Edited by BobGn Thursday, April 6, 2017 4:50 PM
    Monday, April 3, 2017 4:25 PM

All replies

  • Bob,

    A web page is always done with Html (curently HTML5). 

    What you describe are simply what is called textboxes, that is either HTML OR Asp.net HTML bound with JavaScript.

    Visual Studio makes that for you from the design. 

    The VB part in your problem is almost nothing. 

    It can be something if you add to it a database, and then you can create an IO Class. 

    But what is now your problem you can better visit the Microsoft ASP.Net forums.

    http://forums.asp.net

    Be aware that around 1998 people used VB to create HTML code themselves, they don't exist anymore. 


    Success
    Cor

    Monday, April 3, 2017 5:18 PM
  • Thanks for the quick response.

    I have always understood ASP as a server protocol that handles client requests.  I do not have control of the server environment.  I need to be able to push the data and the Submit click to the server from the client.  I have done this before in the VBA environment of a Microsoft Access (2013 edition) application (unassociated with a database), so I was expecting that the VB environment would have these same interfaces

    Given the client side coding need, is your response still applicable? 

    Bob Goodwin


    Bob Goodwin

    Monday, April 3, 2017 6:46 PM
  • Bob,

    A webpage runs in a browser. It is not important which one, they do all the same. Code is rendered by an engine to HTML and JavaScript and that is sent to the webbrowser. As soon as that the user has loged in, a session is created for him. The session keeps him alive as long as they post back data. 

    To do this kind of things with VBA sounds quite impossible, almost as much impossible by using Microsoft Access without a database file. 

    But if you show your VBA code then maybe we can help you to do that in VB for Net. 

    You are right that if it can be done with VBA it could be done with VB for Net. 


    Success
    Cor

    Monday, April 3, 2017 6:57 PM
  • The VBA code, executed within an Access application that accomplished what I am now trying to do in VB.Net, is shown below.  My application does access a database for its primary functions, but it also had a couple of collateral functions that had nothing to do with the database.  The code below is one of those collateral functions that

    a. opens the URL presenting the webpage on the users screen.
    b. pushed a Latitude and Longitude variable into Address1 of the webpage
    c. pushed a City and State into Address2 of the webpage
    d.  activated the submit button

    The webpage then updated its presentation based upon the two addresses.

    Public Sub Plot_Contacts()
        Dim doc As Object
        Dim Form As Object
        
        Set ie = CreateObject("INTERNETEXPLORER.APPLICATION")
        ie.navigate ".....URL...."
        ie.Visible = True
        Do While ie.ReadyState < 4: Loop
        Call SetAddress1 'SetAddress1 loads NCS Lat and Long into Point A input box of HTML code
        
    End Sub
    
    Public Sub SetAddress1()
            'set Point A of Contact Logger to NCS Lat and Long
            ie.Document.getElementsByName("Address1")(0).Value = varLatLong
    End Sub
    
    Public Sub SetAddress2()
            'set Point B of Contact logger to Contact Lat and Long
            ie.Document.getElementsByName("Address2")(0).Value = varCityState
            ie.Document.getElementsByTagName("INPUT")(2).Click
            
    End Sub

    I have been able to open a webpage in VB.Net using

    IEprocess.StartInfo.FileName = "....URL....."

    IEprocess.Start()

    Bob Goodwin


    Bob Goodwin


    • Edited by BobGn Tuesday, April 4, 2017 7:46 PM corrected faulty memory
    Monday, April 3, 2017 7:46 PM
  • Hallo Bob,

    What you show is called currently a webbrowser application. It can be done with C++ without .Net and then you get direct the so called IE information (that is the IE information inside the windows OS not the browser) as it has been in the older Windows systems before Vista.

    As you are now in the VB forum you can use a so called webbrowser control. There are then 3 possibilities

    1. Using Interop
    2. Using the WPF webbrowser control
    3. Using the Windows Forms browser

    Most persons who ask about it here talk about the Windows Forms webbrowser.

    https://msdn.microsoft.com/en-us/library/system.windows.forms.webbrowser(v=vs.110).aspx

    Try it, there is enormous much information about it on Internet. However, be aware it has a limited scope. Trying to hack in whatever way with it is almost impossible. 


    Success
    Cor

    Tuesday, April 4, 2017 8:42 AM
  • The VBA code, executed within an Access application that accomplished what I am now trying to do in VB.Net, is shown below.  My application does access a database for its primary functions, but it also had a couple of collateral functions that had nothing to do with the database.  The code below is one of those collateral functions that

    a. opens the URL presenting the webpage on the users screen.
    b. pushed a Latitude and Longitude variable into Address1 of the webpage
    c. pushed a City and State into Address2 of the webpage
    d.  activated the submit button

    The webpage then updated its presentation based upon the two addresses.

    Public Sub Plot_Contacts()
        Dim doc As Object
        Dim Form As Object
        
        Set ie = CreateObject("INTERNETEXPLORER.APPLICATION")
        ie.navigate ".....URL...."
        ie.Visible = True
        Do While ie.ReadyState < 4: Loop
        Call SetAddress1 'SetAddress1 loads NCS Lat and Long into Point A input box of HTML code
        
    End Sub
    
    Public Sub SetAddress1()
            'set Point A of Contact Logger to NCS Lat and Long
            ie.Document.getElementsByName("Address1")(0).Value = varLatLong
    End Sub
    
    Public Sub SetAddress2()
            'set Point B of Contact logger to Contact Lat and Long
            ie.Document.getElementsByName("Address2")(0).Value = varCityState
            ie.Document.getElementsByTagName("INPUT")(2).Click
            
    End Sub

    I have been able to open a webpage in VB.Net using

    IEprocess.StartInfo.FileName = "....URL....."

    IEprocess.Start()

    Bob Goodwin


    Bob Goodwin


    Hi Bob,

    This isn't the route you'd want to take.  To interact with the webpage you would want to either use a WebBrowser control on a Form and try to interact with the document model, or you could just use an HttpClient instance to post the information to the web form.

    However, it is important to consider the website you want to interact with.  Many sites forbid this kind of interaction from automated software in their terms of use.  Most sites which have data meant for automated consumption offer some kind of Web API to provide that data.

    Can you tell us what site, or sites, you were wanting to work with?


    Reed Kimble - "When you do things right, people won't be sure you've done anything at all"

    Tuesday, April 4, 2017 8:02 PM
    Moderator
  • The website is a hosted site under my control that is running a Google map routine that plots lines on the map between the two addresses that are provided as inputs to the input boxes on the web site.  This is being done automatically by the program based upon contact information that is associated with the contact's Amateur Radio callsign.  The website accepts these inputs from my earlier program using aMicrosoft Access application, but I now want to shift over to a fully executable .exe file that runs my program (a small part of which is to trigger and display the Google maps).

    I have tried to stumble around in the WebBrowser class but have gotten nowhere.  I have a URL.  I need to open it, and then I need to load two values into input text boxes on that website, and submit by sending a 'click' that executes the Submit button for the two input address boxes.


    Bob Goodwin

    Tuesday, April 4, 2017 8:25 PM
  • The website is a hosted site under my control ...

    OK, great!

    Can you make modifications to the site?  The ideal solution would be to add some Web API (or similarly styled) pages to your site so that your desktop application can call a clean set of API methods to interchange data with the website - as opposed to scraping web pages.

    To work with your existing page, the easiest solution is probably to simply post the desired form contents directly to the page.  Don't think about the operations a user would take via a web browser to get the desired result.  Instead, think about the underlying HTTP process which is taking place.  The first three links in that search result should be useful, with the fourth being fully in-depth.

    Here is a rough example which, for clarity, does not take any error handling into account and does not take advantage of the asynchronous nature of the methods used.  This represents the minimal transaction of posting to a web form and retrieving the resulting HTML page.

    Private Function PostAddressInfo(formUrl As String, latLonAddress As String, cityAddress As String) As String
        Using client As New Net.Http.HttpClient
            Dim field1 As New KeyValuePair(Of String, String)("Address1", latLonAddress)
            Dim field2 As New KeyValuePair(Of String, String)("Address2", cityAddress)
            Dim content As New Net.Http.FormUrlEncodedContent({field1, field2})
            Dim response = client.PostAsync(formUrl, content).Result
            Return response.Content.ReadAsStringAsync.Result
        End Using
    End Function

    This code is untested as I do not have the URL to submit to, but I believe it should work.  This method takes the two field values you want to post, along with the URL to post to, submits the form data and then returns the body of the resulting HTML page.

    From there you can either parse the resulting HTML string manually, or attempt to load it into a document model and parse the model.


    Reed Kimble - "When you do things right, people won't be sure you've done anything at all"


    Wednesday, April 5, 2017 3:01 PM
    Moderator