HTTPWebRequest Periodically freezing app.

Question

I have code that I have been using for roughly 2+ years that periodically will freeze my application.  This code runs about every 20 seconds of the day.  It might run perfectly fine for a month with no problems making me think that I have figured out the issue with the code, at other times it causes the program to freeze about ever 2-3 days.  This is very frustrating I would appreciate anyones help in figuring out what is going on.  When the program freezes it's either at the "oRequest.GetRequestStream is Nothing" line or the "oRequest.GetResponse is Nothing" line.

Public Function FiveMin(ByVal cert As String, ByVal pass As String, ByVal location As String, ByVal type As String, ByVal xsdfile As String, ByVal table As String, ByVal field As String)

        Dim BUFFERLENGTH As Integer = 1024
        Dim buffer(BUFFERLENGTH) As Byte
        Dim size As Integer = 0
        Dim bytesRead As Integer = 0
        Dim sData, xmlstring As String
        Dim store As X509Store
        Dim certificates As X509Certificate2Collection
        Dim certificate As X509Certificate
        Dim oRequest As HttpWebRequest
        Dim dataStream As Stream
        Dim MemoryStream As MemoryStream
        Dim StreamWriter As StreamWriter
        Dim oResponse As WebResponse
        Dim StreamReader As StreamReader
        Dim bArray As Byte()

        'xml query
        sData = "<?xml version=""1.0"" encoding=""utf-8""?>" _
                         & "<soap:Envelope xmlns:soap=""http://schemas.xmlsoap.org/soap/envelope/"">" _
                             & "<soap:Body>" _
                                    & "<QueryRequest xmlns=""http://markets.abc.com/art/xml"">" _
                                       & "<QueryRealTimeLMP type=""" + type + """>" _
                                            & "<LocationName>" + location + "</LocationName>" _
                                       & "</QueryRealTimeLMP>" _
                                    & "</QueryRequest>" _
                             & "</soap:Body>" _
                         & "</soap:Envelope>"

        'Get Certificate from the Certificate Store on the local Machine
        store = New X509Store(StoreName.My, StoreLocation.LocalMachine)
        store.Open(OpenFlags.ReadOnly)
        certificates = store.Certificates.Find(X509FindType.FindBySubjectName, cert, True)
        certificate = certificates(0)

        oRequest = DirectCast(WebRequest.Create("https://markets.abc.com/art/xml/query"), HttpWebRequest)
        oRequest.ClientCertificates.Add(certificate)
        oRequest.Method = "POST"
        'convert data to a byte array
        bArray = Encoding.ASCII.GetBytes(sData)
        'set content type
        oRequest.ContentType = "text/xml"
        oRequest.Headers.Add("SOAPAction", "/art/xml/query")
        ' Set the ContentLength property of the WebRequest.

        oRequest.Timeout = 10000
        oRequest.KeepAlive = False
              ' Get the request stream.

        MemoryStream = New MemoryStream
        StreamWriter = New StreamWriter(MemoryStream)
        StreamReader = New StreamReader(MemoryStream)

        Try
            If oRequest.GetRequestStream Is Nothing Then
                oRequest.GetRequestStream.Close()
                oRequest.GetRequestStream.Dispose()
                Return False
            Else
                dataStream = oRequest.GetRequestStream
            End If

            ' Write the data to the request stream.
            dataStream.Write(bArray, 0, bArray.Length)
            ' Close the Stream object.
            dataStream.Close()
            dataStream.Dispose()
            ' Get the response.
            If oRequest.GetResponse Is Nothing Then
                oRequest.GetResponse.Close()
                Return False
            Else
                oResponse = oRequest.GetResponse()
            End If

            dataStream = oResponse.GetResponseStream()
      
            While size <> -1
                size = dataStream.Read(buffer, 0, BUFFERLENGTH)
                If size = 0 Then Exit While
                bytesRead += size
                If size < BUFFERLENGTH Then
                    Dim buff(size) As Byte
                    Array.Copy(buffer, buff, size)
                    MemoryStream.Write(buff, 0, size)
                    buff = Nothing
                Else
                    MemoryStream.Write(buffer, 0, size)
                End If
                Array.Clear(buffer, 0, size)
            End While
          
            MemoryStream.Position = 0

            xmlstring = StreamReader.ReadToEnd
            If validatexml(xsdfile, xmlstring) = True Then
                FiveMinImport(xmlstring, table, field)
            Else
                Return False
            End If
            oResponse.Close()
            oRequest.Abort()
            dataStream.Close()

        Catch VEex As ProtocolViolationException
            oRequest.Abort()
            Return False
        Catch ex As NullReferenceException
            oRequest.Abort()
            Return False
        Catch webex As WebException
            oRequest.Abort()
            Return False
        Catch ioex As IOException
            oRequest.Abort()
            Return False
        Finally

            oResponse.Close()
            MemoryStream.Close()
            MemoryStream.Dispose()
            StreamWriter.Flush()
            StreamWriter.Close()
            StreamWriter.Dispose()
            StreamReader.Close()
            StreamReader.Dispose()
            oRequest.Abort()
            dataStream.Close()
        End Try
        oRequest.Abort()
        Return DLCheck
    End Function

Any other mistakes in my code I would appreciate it if you could point it out.

Answer 1

Hi,

Welcome to MSDN.

It depends on a lot things, if this issue is urgent, please contact support directly(http://support.microsoft.com)

You could have a look at the following suggestions:

1. According to the following words of Chapter 14. Networking:

Warning
If you don't have a proxy, you must set the Proxy property to null on all WebClient and WebRequest objects. Otherwise, the          Framework may attempt to "auto-detect" your proxy settings, adding          up to 30 seconds to your request. If you're wondering why your web          requests execute slowly, this is probably it!

I would suggest you add this line to your code to test:

 oRequest = DirectCast(WebRequest.Create("https://markets.abc.com/art/xml/query"), HttpWebRequest)
        oRequest.Proxy = Nothing

2. If still no helping, you set oRequest and oResponse to Nothing in Finally to test.

3. In order to make sure the previous request has been released successfully, you could add this line in first line of that method, then change DefaultConnectionLimit to a bigger number as following.

    Public Function FiveMin(ByVal cert As String, ByVal pass As String, ByVal location As String, ByVal type As String, ByVal xsdfile As String, ByVal table As String, ByVal field As String)
        System.GC.Collect()
        System.Net.ServicePointManager.DefaultConnectionLimit = 200

In addition, if you are using previous .Net Framework of .Net 4.0, I would suggest you consider changing it to .Net 4.0 or .Net 4.5 to test.

Since this issue maybe related to .Net Framework, you could also consider posting this issue in the following forum to get supports.

.NET Framework forums>.NET Framework Class Libraries

Regards.

---

We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
Click HERE to participate the survey.

Answer 2

Maybe should you refactor your code a little bit.

It looks very old fashion and therefore not easy to maintain (and that means also to understand by others)

You first start with declaring and at the end you are handling things like if they don't go out of scope.

Be aware adding all kind of code just to be sure is a beginners way which does not make it more stable, also remove code which in fact does nothing to avoid it goes hanging on that.

---

Success
Cor

Answer 3

In the proxy area of your reply it states that the Framework can attempt to auto-detect the proxy settings for up to 30 seconds.  However when my app freezes/gets caught in a loop at either or the 2 lines I specified it stays there indefinately...Thus causing the app to basically freeze.

I will implement your suggested changes, mark the question as answered for the time being.   

Answer 4

In the proxy area of your reply it states that the Framework can attempt to auto-detect the proxy settings for up to 30 seconds.  However when my app freezes/gets caught in a loop at either or the 2 lines I specified it stays there indefinately...Thus causing the app to basically freeze.

I will implement your suggested changes, mark the question as answered for the time being.   

I'm not a real function guru but it seems to me you don't need to catch specific exceptions since you don't do some kind of notification about them. Also it seems to me some of your "If - End If" statements were unnecessary.

I don't know if this code will be any better. Also my VS requires a data type declaration at the end of the function declaration or something.

And the bit of code in the first code block below maybe the FiveMinImport function or sub is causing the issue.

If validatexml(xsdfile, xmlstring) = True Then
      FiveMinImport(xmlstring, table, field)
Else
      Return False
End If

Function - I doubt the "Exit Function" is required in the try catch to keep from trying to return DLCheck. And I just removed the If then else statements for streams being nothing. As I would think the Try Catch would go into effect returning False for those.

Public Function FiveMin(ByVal cert As String, ByVal pass As String, ByVal location As String, ByVal type As String, ByVal xsdfile As String, ByVal table As String, ByVal field As String) As Boolean

        Dim BUFFERLENGTH As Integer = 1024
        Dim buffer(BUFFERLENGTH) As Byte
        Dim size As Integer = 0
        Dim bytesRead As Integer = 0
        Dim sData, xmlstring As String
        Dim store As X509Store
        Dim certificates As X509Certificate2Collection
        Dim certificate As X509Certificate
        Dim oRequest As HttpWebRequest
        Dim dataStream As Stream
        Dim MemoryStream As MemoryStream
        Dim StreamWriter As StreamWriter
        Dim oResponse As WebResponse
        Dim StreamReader As StreamReader
        Dim bArray As Byte()

        Try
            'xml query
            sData = "<?xml version=""1.0"" encoding=""utf-8""?>" _
                             & "<soap:Envelope xmlns:soap=""http://schemas.xmlsoap.org/soap/envelope/"">" _
                                 & "<soap:Body>" _
                                        & "<QueryRequest xmlns=""http://markets.abc.com/art/xml"">" _
                                           & "<QueryRealTimeLMP type=""" + type + """>" _
                                                & "<LocationName>" + location + "</LocationName>" _
                                           & "</QueryRealTimeLMP>" _
                                        & "</QueryRequest>" _
                                 & "</soap:Body>" _
                             & "</soap:Envelope>"

            'Get Certificate from the Certificate Store on the local Machine
            store = New X509Store(StoreName.My, StoreLocation.LocalMachine)
            store.Open(OpenFlags.ReadOnly)
            certificates = store.Certificates.Find(X509FindType.FindBySubjectName, cert, True)
            certificate = certificates(0)

            oRequest = DirectCast(WebRequest.Create("https://markets.abc.com/art/xml/query"), HttpWebRequest)
            oRequest.ClientCertificates.Add(certificate)
            oRequest.Method = "POST"
            'convert data to a byte array
            bArray = Encoding.ASCII.GetBytes(sData)
            'set content type
            oRequest.ContentType = "text/xml"
            oRequest.Headers.Add("SOAPAction", "/art/xml/query")
            ' Set the ContentLength property of the WebRequest.

            oRequest.Timeout = 10000
            oRequest.KeepAlive = False
            ' Get the request stream.

            MemoryStream = New MemoryStream
            StreamWriter = New StreamWriter(MemoryStream)
            StreamReader = New StreamReader(MemoryStream)


            dataStream = oRequest.GetRequestStream
                ' Write the data to the request stream.
                dataStream.Write(bArray, 0, bArray.Length)
                ' Close the Stream object.
                dataStream.Close()
                dataStream.Dispose()
            ' Get the response.
            oResponse = oRequest.GetResponse()
                dataStream = oResponse.GetResponseStream()
                While size <> -1
                    size = dataStream.Read(buffer, 0, BUFFERLENGTH)
                    If size = 0 Then Exit While
                    bytesRead += size
                    If size < BUFFERLENGTH Then
                        Dim buff(size) As Byte
                        Array.Copy(buffer, buff, size)
                        MemoryStream.Write(buff, 0, size)
                        buff = Nothing
                    Else
                        MemoryStream.Write(buffer, 0, size)
                    End If
                    Array.Clear(buffer, 0, size)
                End While

                MemoryStream.Position = 0

                xmlstring = StreamReader.ReadToEnd
                If validatexml(xsdfile, xmlstring) = True Then
                    FiveMinImport(xmlstring, table, field)
                Else
                    Return False
                End If
                oResponse.Close()
                MemoryStream.Close()
                MemoryStream.Dispose()
                StreamWriter.Flush()
                StreamWriter.Close()
                StreamWriter.Dispose()
                StreamReader.Close()
                StreamReader.Dispose()
                oRequest.Abort()
                dataStream.Close()
        Catch ex As Exception
            oResponse.Close()
            MemoryStream.Close()
            MemoryStream.Dispose()
            StreamWriter.Flush()
            StreamWriter.Close()
            StreamWriter.Dispose()
            StreamReader.Close()
            StreamReader.Dispose()
            oRequest.Abort()
            dataStream.Close()
            Return False
            Exit Function
        End Try
        Return DLCheck
    End Function

Or maybe this with Return DLCheck before the catch of the try catch if that's allowed.

Public Function FiveMin(ByVal cert As String, ByVal pass As String, ByVal location As String, ByVal type As String, ByVal xsdfile As String, ByVal table As String, ByVal field As String) As Boolean

        Dim BUFFERLENGTH As Integer = 1024
        Dim buffer(BUFFERLENGTH) As Byte
        Dim size As Integer = 0
        Dim bytesRead As Integer = 0
        Dim sData, xmlstring As String
        Dim store As X509Store
        Dim certificates As X509Certificate2Collection
        Dim certificate As X509Certificate
        Dim oRequest As HttpWebRequest
        Dim dataStream As Stream
        Dim MemoryStream As MemoryStream
        Dim StreamWriter As StreamWriter
        Dim oResponse As WebResponse
        Dim StreamReader As StreamReader
        Dim bArray As Byte()

        Try
            'xml query
            sData = "<?xml version=""1.0"" encoding=""utf-8""?>" _
                             & "<soap:Envelope xmlns:soap=""http://schemas.xmlsoap.org/soap/envelope/"">" _
                                 & "<soap:Body>" _
                                        & "<QueryRequest xmlns=""http://markets.abc.com/art/xml"">" _
                                           & "<QueryRealTimeLMP type=""" + type + """>" _
                                                & "<LocationName>" + location + "</LocationName>" _
                                           & "</QueryRealTimeLMP>" _
                                        & "</QueryRequest>" _
                                 & "</soap:Body>" _
                             & "</soap:Envelope>"

            'Get Certificate from the Certificate Store on the local Machine
            store = New X509Store(StoreName.My, StoreLocation.LocalMachine)
            store.Open(OpenFlags.ReadOnly)
            certificates = store.Certificates.Find(X509FindType.FindBySubjectName, cert, True)
            certificate = certificates(0)

            oRequest = DirectCast(WebRequest.Create("https://markets.abc.com/art/xml/query"), HttpWebRequest)
            oRequest.ClientCertificates.Add(certificate)
            oRequest.Method = "POST"
            'convert data to a byte array
            bArray = Encoding.ASCII.GetBytes(sData)
            'set content type
            oRequest.ContentType = "text/xml"
            oRequest.Headers.Add("SOAPAction", "/art/xml/query")
            ' Set the ContentLength property of the WebRequest.

            oRequest.Timeout = 10000
            oRequest.KeepAlive = False
            ' Get the request stream.

            MemoryStream = New MemoryStream
            StreamWriter = New StreamWriter(MemoryStream)
            StreamReader = New StreamReader(MemoryStream)


            dataStream = oRequest.GetRequestStream
            ' Write the data to the request stream.
            dataStream.Write(bArray, 0, bArray.Length)
            ' Close the Stream object.
            dataStream.Close()
            dataStream.Dispose()
            ' Get the response.
            oResponse = oRequest.GetResponse()
            dataStream = oResponse.GetResponseStream()
            While size <> -1
                size = dataStream.Read(buffer, 0, BUFFERLENGTH)
                If size = 0 Then Exit While
                bytesRead += size
                If size < BUFFERLENGTH Then
                    Dim buff(size) As Byte
                    Array.Copy(buffer, buff, size)
                    MemoryStream.Write(buff, 0, size)
                    buff = Nothing
                Else
                    MemoryStream.Write(buffer, 0, size)
                End If
                Array.Clear(buffer, 0, size)
            End While

            MemoryStream.Position = 0

            xmlstring = StreamReader.ReadToEnd
            If validatexml(xsdfile, xmlstring) = True Then
                FiveMinImport(xmlstring, table, field)
            Else
                Return False
            End If
            oResponse.Close()
            MemoryStream.Close()
            MemoryStream.Dispose()
            StreamWriter.Flush()
            StreamWriter.Close()
            StreamWriter.Dispose()
            StreamReader.Close()
            StreamReader.Dispose()
            oRequest.Abort()
            dataStream.Close()
            Return DLCheck
        Catch ex As Exception
            oResponse.Close()
            MemoryStream.Close()
            MemoryStream.Dispose()
            StreamWriter.Flush()
            StreamWriter.Close()
            StreamWriter.Dispose()
            StreamReader.Close()
            StreamReader.Dispose()
            oRequest.Abort()
            dataStream.Close()
            Return False
            Exit Function
        End Try
    End Function

---

Please BEWARE that I have NO EXPERIENCE and NO EXPERTISE and probably onset of DEMENTIA which may affect my answers! Also, I've been told by an expert, that when you post an image it clutters up the thread and mysteriously, over time, the link to the image will somehow become "unstable" or something to that effect. :) I can only surmise that is due to Global Warming of the threads.