Rare exception: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host

Question

Our .NET 2.0 app relies upon the ability to download xml webpages from URLs.  At startup, we have about 6 boxes in the wild (out of thousands) that are seeing the following inner exception when they try to download an xml page. 

System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive.

(Inner: ) System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.

They can get to the URL in their browser just fine, so we know our server is working.  Furthermore, the embedded WebBrowser control in our app can pull up webpages just fine, so I don't think their firewall is blocking our app.  I'm hoping it's just some diagnosible configuration issue. 

Here is the snippet of code that does the downloading:

WebRequest request = WebRequest.Create(fileToDownload.SourceUri);
request.CachePolicy = new RequestCachePolicy(RequestCacheLevel.NoCacheNoStore);
request.Timeout = (int)DefaultTimeout.TotalMilliseconds;
request.Method = "HEAD";
WebResponse response = request.GetResponse();

Nothing revolutionary there. (I've also tried it without the HEAD request, without keepalive, and using HTTP 1.0).  And here is the exception:

System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive.

StackTrace Information Details:
======================================
   at System.Net.HttpWebRequest.GetResponse()
   ...Our code...

Inner Exception: System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.
StackTrace Information Details:
======================================
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.PooledStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.Connection.SyncRead(HttpWebRequest request, Boolean userRetrievedStream, Boolean probeRead)

And lastly, here is the network log:

System.Net Verbose: 0 : [3704] WebRequest::Create(http://launcher.patcher.ncsoft.com/NCLauncher/Main/Manifests/CurrentVersion.xml)
System.Net Verbose: 0 : [3704] HttpWebRequest#48209832::HttpWebRequest(http://launcher.patcher.ncsoft.com/NCLauncher/Main/Manifests/CurrentVersion.xml#-26695995)
System.Net Verbose: 0 : [3704] Exiting HttpWebRequest#48209832::HttpWebRequest()
System.Net Verbose: 0 : [3704] Exiting WebRequest::Create()  -> HttpWebRequest#48209832
System.Net Verbose: 0 : [3704] HttpWebRequest#48209832::GetResponse()
System.Net Information: 0 : [3704] Associating HttpWebRequest#48209832 with ServicePoint#5773521
System.Net.Cache Verbose: 0 : [3704] RequestCacheProtocol#54135081::GetRetrieveStatus(HttpWebRequest#48209832)
System.Net.Cache Information: 0 : [3704] Request#48209832, Policy = Level:NoCacheNoStore, Cache Uri = http://launcher.patcher.ncsoft.com/NCLauncher/Main/Manifests/CurrentVersion.xml
System.Net.Cache Information: 0 : [3704] Request Method = HEAD.
System.Net.Cache Information: 0 : [3704] Selected cache Key = http://launcher.patcher.ncsoft.com/NCLauncher/Main/Manifests/CurrentVersion.xml
System.Net.Cache Verbose: 0 : [3704] Exiting RequestCacheProtocol#54135081::GetRetrieveStatus()  -> result = DoNotTakeFromCache
System.Net Information: 0 : [3704] Associating Connection#63094882 with HttpWebRequest#48209832
System.Net.Sockets Verbose: 0 : [3704] Socket#49924125::Socket(InterNetwork#2)
System.Net.Sockets Verbose: 0 : [3704] Exiting Socket#49924125::Socket()
System.Net.Sockets Verbose: 0 : [3704] Socket#49924125::Connect(204:80#-862290018)
System.Net.Sockets Verbose: 0 : [3704] Exiting Socket#49924125::Connect()
System.Net Information: 0 : [3704] Associating HttpWebRequest#48209832 with ConnectStream#41773672
System.Net Information: 0 : [3704] HttpWebRequest#48209832 - Request: HEAD /NCLauncher/Main/Manifests/CurrentVersion.xml HTTP/1.1

System.Net.Sockets Verbose: 0 : [3704] Socket#49924125::Send()
System.Net.Sockets Verbose: 0 : [3704] Data from Socket#49924125::Send
System.Net.Sockets Verbose: 0 : [3704] 00000000 : 48 45 41 44 20 2F 4E 43-4C 61 75 6E 63 68 65 72 : HEAD /NCLauncher
System.Net.Sockets Verbose: 0 : [3704] 00000010 : 2F 4D 61 69 6E 2F 4D 61-6E 69 66 65 73 74 73 2F : /Main/Manifests/
System.Net.Sockets Verbose: 0 : [3704] 00000020 : 43 75 72 72 65 6E 74 56-65 72 73 69 6F 6E 2E 78 : CurrentVersion.x
System.Net.Sockets Verbose: 0 : [3704] 00000030 : 6D 6C 20 48 54 54 50 2F-31 2E 31 0D 0A 48 6F 73 : ml HTTP/1.1..Hos
System.Net.Sockets Verbose: 0 : [3704] 00000040 : 74 3A 20 6C 61 75 6E 63-68 65 72 2E 70 61 74 63 : t: launcher.patc
System.Net.Sockets Verbose: 0 : [3704] 00000050 : 68 65 72 2E 6E 63 73 6F-66 74 2E 63 6F 6D 0D 0A : her.ncsoft.com..
System.Net.Sockets Verbose: 0 : [3704] 00000060 : 43 61 63 68 65 2D 43 6F-6E 74 72 6F 6C 3A 20 6E : Cache-Control: n
System.Net.Sockets Verbose: 0 : [3704] 00000070 : 6F 2D 73 74 6F 72 65 2C-6E 6F 2D 63 61 63 68 65 : o-store,no-cache
System.Net.Sockets Verbose: 0 : [3704] 00000080 : 0D 0A 50 72 61 67 6D 61-3A 20 6E 6F 2D 63 61 63 : ..Pragma: no-cac
System.Net.Sockets Verbose: 0 : [3704] 00000090 : 68 65 0D 0A 43 6F 6E 6E-65 63 74 69 6F 6E 3A 20 : he..Connection:
System.Net.Sockets Verbose: 0 : [3704] 000000A0 : 4B 65 65 70 2D 41 6C 69-76 65 0D 0A 0D 0A       : Keep-Alive....
System.Net.Sockets Verbose: 0 : [3704] Exiting Socket#49924125::Send()  -> 174#174
System.Net Information: 0 : [3704] ConnectStream#41773672 - Sending headers
{
Host: launcher.patcher.ncsoft.com
Cache-Control: no-store,no-cache
Pragma: no-cache
Connection: Keep-Alive
}.
System.Net.Sockets Verbose: 0 : [3704] Socket#49924125::Receive()
System.Net.Sockets Error: 0 : [3704] Exception in the Socket#49924125::Receive - An existing connection was forcibly closed by the remote host
System.Net.Sockets Verbose: 0 : [3704] Exiting Socket#49924125::Receive()  -> 0#0
System.Net.Sockets Verbose: 0 : [3704] Socket#49924125::Dispose()
System.Net Error: 0 : [3704] Exception in the HttpWebRequest#48209832:: - The underlying connection was closed: An unexpected error occurred on a receive.

 

I'm pretty darn sure our web server isn't closing the connection, since, like I say, they can browse to the URL in their browser just fine.  Could this be some sort of strange proxy server issue?  I tried disabling proxy connections on one of the customer's machines (by setting WebRequest.DefaultWebProxy to null, but that doesn't seem to make a difference either.

Any ideas - even shots in the dark - would be appreciated!

Many thanks,

Taylor

Answer 1

One of the customers that has this problem *used* to be able to connect to our servers with our app.  He claims that the only thing he changed was the internet connection.  Whereas previously he was using an open wireless connection at school, he is now going through his home router.  He can still browse the the URL just fine in his browser though, so his internet connection is working.  I can't imagine something like this affecting our app, but I'm going to try to do more network tracing to see.

Has anyone ever heard of this? I have seen a number of posts about this bug popping up in strange places, but I never see a solution...That can't bode well. :)

Many thanks for any help,

Taylor

Answer 2

I had our customer netcap the data, and then I loaded it in Ethereal.  it shows the following:

21.613281    192.168.1.64          206.127.154.204  TCP   1117 > http [SYN] Seq=0 Len=0 MSS=1460

21.953125    206.127.154.204  192.168.1.64          TCP    http > 1117 [SYN, ACK] Seq=0 Ack=1 Win=8190 Len=0 MSS=1452

21.952125    192.168.1.64          206.127.154.204  TCP    1117 > http [ACK] Seq=1 Ack=1 Win=65535 Len=0

21.952125    192.168.1.64          206.127.154.204  HTTP    GET /NCLauncher/Main/Manifests/CurrentVersion.xml  HTTP/1.1

21.966797     206.127.154.204  192.168.1.64         TCP    http > 1117 [RST] Seq=1 Len=0

In other words, the web server talks back to the client just fine until the GET request is made.  Then, immediately, the connection is reset.  It happens so fast that this must only be a client-side thing.  Is this what it would look like if a firewall blocked the outbound connection?  What else might cause a connection to close so suddenly like this?

Many thanks,

Taylor

Answer 3

Problem solved!

 

All of these users are in Singapore.  Apparently either their ISP, or the router they give to their customers, filters HTTP GET requests.  In particular, it is requiring that a GET request have either one of the following headers:

"Accept:*/*"

or

"User-Agent:<Insert user agent here>"

 

If I add either one of those headers to my web requests, it works like a champ.  For that reason, I now add an "Accept:*/*" header to ALL of my GET requests:

webRequest.Accept = "*/*";

 

Hopefully this will be useful to someone else as well!

Answer 4

How do I incorporate the fix in generated web service proxy class?

thanks and regards,

Answer 5

 I just struggled with that one for 2 days. Thanks for the info.

Answer 6

Hi Anch1ang,  did you find any solution for this? I want to incorporate the same in the proxy class.

Answer 7

"An existing connection was forcibly closed by the remote host" can also be generated on a request where the server is compressing all response data by default.

In this situation, it's neccessary to add the following request header:

"Accept-Encoding", "gzip, deflate"

To decompress the resulting response, use System.IO.Compression.GZipStream.

 

 

 

 

Answer 8

Hi All,

For those of you still fighting with this issue and cannot control the host servers here is the solution: 

Problem = Host server closes the connection prematurely on HttpWebRequest calls

Solution = Get the Response Stream into a local byte[] as quickly as possible and read from there

Code = 

int b1;

memoryStream = new MemoryStream();

while ((b1 = httpWebResponse.GetResponseStream().ReadByte()) != -1) { memoryStream.WriteByte(((byte)b1)); }

byte[] xmlBytes = memoryStream.ToArray();

httpWebResponse.Close();

memoryStream.Close();

memoryStream = new MemoryStream(xmlBytes);

...do your thing with the local MemoryStream (don't forget to close it)

 

Hope this helps.

 

Answer 9

Hi,

I am facing same problem. I tried all the options available above. It is happening with one wifi network only.

Following is my code:

HttpWebRequest

 

 

myHttpWebRequest = Create(http://www.yahoo.com);

myHttpWebRequest.Method = "GET";

myHttpWebRequest.AllowAutoRedirect =

 

false;

 

 

// myHttpWebRequest.UserAgent = userAgent;

myHttpWebRequest.Timeout = 20000;

myHttpWebRequest.Accept =

 

"*/*";

 

 

// Sends the HttpWebRequest and waits for the response.

 

 

HttpWebResponse myHttpWebResponse = (HttpWebResponse)myHttpWebRequest.GetResponse();

 

 

 

Stream receiveStream = myHttpWebResponse.GetResponseStream();

 

 

Encoding encode = System.Text.Encoding.GetEncoding("utf-8");

 

 

// Pipes the stream to a higher level stream reader with the required encoding format.

 

 

StreamReader readStream = new StreamReader(receiveStream, encode);

 

 

Console.WriteLine("\r\nResponse stream received.");

 

 

Char[] read = new Char[1024];

 

 

// Reads 256 characters at a time.

 

 

int count = readStream.Read(read, 0, 1024);

 

 

Console.WriteLine("HTML...\r\n");

 

 

try

{

 

 

while (count > 0)

{

 

 

// Dumps the 256 characters on a string and displays the string to the console.

 

 

String str = new String(read, 0, count);

 

 

Console.Write(str);

count = readStream.Read(read, 0, 1024);

}

}

 

 

catch (Exception ex)

{

 

 

string str = ex.Message;

 

 

Console.Write("Exception {0}", str);

}

 

 

Console.WriteLine("");

 

 

// Releases the resources of the response.

myHttpWebResponse.Close();

 

 

// Releases the resources of the Stream.

readStream.Close();

 

StackTarce

  at System.Net.ConnectStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.IO.StreamReader.ReadBuffer(Char[] userBuffer, Int32 userOffset, Int32 desiredChars, Boolean& readToUserBuffer)
   at System.IO.StreamReader.Read(Char[] buffer, Int32 index, Int32 count)
   at TestApp.MyClass.MYFunction() in C:\Project\Examples\TestApp\TestApp\checkInternet.cs:line 89

Exception Message: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.

 

If any one has solution, please do update here.

Thanks in advance.

Raj

Answer 10

What is this script doing exactly, I am facing the same type of issue thx.

---

JCtech1123, Cheers