locked
VB.NET WebClient Proxy Issue... RRS feed

  • Question

  • I have been working on a small application that can crawl Google to show some of the search engine results pages and the WebClient normally connects fine, but after trying to connect to Google thru the WebClient proxy property, it just isn't connecting properly. Here is the code that I'm working with:

    Try
                Dim wc As New System.Net.WebClient()
               'This line below will bring in the "User Agent" to make the HTTP request:
                wc.Headers.Add("user-agent", "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36")               
                Dim wp As New System.Net.WebProxy(TextBox1.Text)
                wc.Proxy = wp
                Dim HTML As String = wc.DownloadString("https://www.google.com/search?q=dog+training")
    
            Catch ex As Exception
                MsgBox(ex.Message)
    End Try

    I know that I have found proxies that should work with Google as I've clearly tested them in my Google Chrome browser to see if they're dead or alive, but when I try to connect in my own application through the WebClient proxy setup, I keep getting the following error messages:

    "Unable to connect to the remote server"

    OR

    "(503) Server Unavailable"

    What am I missing? Could it be the headers are not fully configured or some other properties? Would appreciate any ideas.


    Sunday, January 7, 2018 5:31 PM

All replies

  • What I did was, edit app.config

    <?xml version="1.0" encoding="utf-8" ?>
    <configuration>
        <startup>
            <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.6.1" />
        </startup>
      <system.net>
        <defaultProxy enabled="true" useDefaultCredentials="true">
          <proxy usesystemdefault="True" />
        </defaultProxy>
      </system.net>  
    </configuration>

    Code

    Imports System.Net
    
    Public Class Form1
        Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
            Try
                Dim wc As New System.Net.WebClient()
                'This line below will bring in the "User Agent" to make the HTTP request:
                wc.Headers.Add("user-agent", "Mozilla/5.0 (Windows NT 6.1; Win64; x64) " &
                               "AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36")
                Dim wp As New WebProxy
                wc.Proxy = wp
                Dim HTML As String = wc.DownloadString("https://www.google.com/search?q=dog+training")
                MessageBox.Show("done")
            Catch ex As Exception
                MessageBox.Show(ex.Message)
            End Try
        End Sub
    End Class


    Please remember to mark the replies as answers if they help and unmark them if they provide no help, this will help others who are looking for solutions to the same or similar problem. Contact via my Twitter (Karen Payne) or Facebook (Karen Payne) via my MSDN profile but will not answer coding question on either.
    VB Forums - moderator
    profile for Karen Payne on Stack Exchange, a network of free, community-driven Q&A sites



    Sunday, January 7, 2018 5:51 PM
  • Hi Karen and thank you for your reply.

    Is this XML written directly into the application or did you change a file elsewhere for the Chrome browser settings? I would need the proxy to be set inside of the vb application and change it periodically as needed. I don't see in your 2 lines of code  where to set the proxy address & port numbers:

    Dim wp As New WebProxy
    wc.Proxy = wp

    So again, it would be nice if I could configure my proxy within a textbox control and have the ability to change it periodically. How would I set this up using this code? I don't see any option with the way that you have this set at this point. Thanks again.

    Sunday, January 7, 2018 7:54 PM
  • I did this within app.config

    That is what the majority of developer use (also in web apps too), see also.

    You can look too 

    GlobalProxySelection.Select = New WebProxy() ' 12 overloads

    Or this StackOverflow post (in C# but here is the VB.NET version below)

    Namespace SomeNameSpace
        Public Class MyProxy
            Implements IWebProxy
    
            Public Property Credentials() As ICredentials
                Get
                    Return New NetworkCredential("user", "password")
                End Get
                'or get { return new NetworkCredential("user", "password","domain"); }
                Set(ByVal value As ICredentials)
                End Set
            End Property
    
            Private Property IWebProxy_Credentials As ICredentials Implements IWebProxy.Credentials
                Get
                    Throw New NotImplementedException()
                End Get
                Set(value As ICredentials)
                    Throw New NotImplementedException()
                End Set
            End Property
    
            Public Function GetProxy(ByVal destination As Uri) As Uri
                Return New Uri("http://my.proxy:8080")
            End Function
    
            Public Function IsBypassed(ByVal host As Uri) As Boolean
                Return False
            End Function
    
            Private Function IWebProxy_GetProxy(destination As Uri) As Uri Implements IWebProxy.GetProxy
                'Throw New NotImplementedException()
            End Function
    
            Private Function IWebProxy_IsBypassed(host As Uri) As Boolean Implements IWebProxy.IsBypassed
                'Throw New NotImplementedException()
            End Function
        End Class
    End Namespace


    Please remember to mark the replies as answers if they help and unmark them if they provide no help, this will help others who are looking for solutions to the same or similar problem. Contact via my Twitter (Karen Payne) or Facebook (Karen Payne) via my MSDN profile but will not answer coding question on either.
    VB Forums - moderator
    profile for Karen Payne on Stack Exchange, a network of free, community-driven Q&A sites

    Sunday, January 7, 2018 8:25 PM