locked
HttpWebRequest.GetRequestStream() hangs / bug in System.Net stack ? RRS feed

  • Question

  • Hi guys,

     

    I have a problem occuring randomly in a scenario of HTTP PUT over HTTPS using the HttpWebRequest . GetRequestStream () API. Some network problem may cause the thread to hang and wait forever.

    Although it has a quite rare occurence, I've managed to reproduce the problem with a ipconfig /release while the thread was trying to do its PUT. The thread got stuck here (forever) :

     

     	[In a sleep, wait, or join]
    	mscorlib.dll!System.Threading.WaitHandle.WaitOne(long timeout, bool exitContext) + 0x2f bytes	
     	mscorlib.dll!System.Threading.WaitHandle.WaitOne(int millisecondsTimeout, bool exitContext) + 0x25 bytes	
     	System.dll!System.Net.LazyAsyncResult.WaitForCompletion(bool snap) + 0xd3 bytes	
     	System.dll!System.Net.Security.SslState.CheckEnqueueRead(byte[] buffer = {byte[4096]}, int offset = 0, int count = 4096, System.Net.AsyncProtocolRequest request) + 0x194 bytes
     	System.dll!System.Net.Security._SslStream.StartReading(byte[] buffer = {byte[4096]}, int offset = 0, int count = 4096, System.Net.AsyncProtocolRequest asyncRequest = null) + 0x6d bytes	
     	System.dll!System.Net.Security._SslStream.ProcessRead(byte[] buffer, int offset, int count, System.Net.AsyncProtocolRequest asyncRequest = null) + 0x6b bytes	
     	System.dll!System.Net.TlsStream.Read(byte[] buffer, int offset, int size) + 0x58 bytes	
     	System.dll!System.Net.PooledStream.Read(byte[] buffer, int offset, int size) + 0x1b bytes	
     	System.dll!System.Net.Connection.SyncRead(System.Net.HttpWebRequest request = {System.Net.HttpWebRequest}, bool userRetrievedStream = false, bool probeRead = true) + 0x12a bytes	
     	System.dll!System.Net.Connection.PollAndRead(System.Net.HttpWebRequest request, bool userRetrievedStream) + 0x5a bytes	
     	System.dll!System.Net.ConnectStream.PollAndRead(bool userRetrievedStream) + 0x1b bytes	
     	System.dll!System.Net.HttpWebRequest.EndWriteHeaders(bool async) + 0xa2 bytes	
     	System.dll!System.Net.HttpWebRequest.WriteHeadersCallback(System.Net.WebExceptionStatus errorStatus, System.Net.ConnectStream stream = {System.Net.ConnectStream}, bool async) + 0x16 bytes	
     	System.dll!System.Net.ConnectStream.WriteHeaders(bool async) + 0x2d1 bytes	
     	System.dll!System.Net.HttpWebRequest.EndSubmitRequest() + 0x82 bytes	
     	System.dll!System.Net.HttpWebRequest.SetRequestSubmitDone(System.Net.ConnectStream submitStream) + 0xf7 bytes	
     	System.dll!System.Net.Connection.CompleteConnection(bool async, System.Net.HttpWebRequest request = {System.Net.HttpWebRequest}) + 0x158 bytes	
     	System.dll!System.Net.Connection.CompleteStartConnection(bool async, System.Net.HttpWebRequest httpWebRequest) + 0x177 bytes	
     	System.dll!System.Net.Connection.CompleteStartRequest(bool onSubmitThread, System.Net.HttpWebRequest request = {System.Net.HttpWebRequest}, System.Net.TriState needReConnect = True) + 0x9a bytes	
     	System.dll!System.Net.Connection.SubmitRequest(System.Net.HttpWebRequest request = {System.Net.HttpWebRequest}) + 0x293 bytes	
     	System.dll!System.Net.ServicePoint.SubmitRequest(System.Net.HttpWebRequest request = {System.Net.HttpWebRequest}, string connName = "S>1054081937") + 0x7c bytes	
     	System.dll!System.Net.HttpWebRequest.SubmitRequest(System.Net.ServicePoint servicePoint) + 0xf9 bytes	
     	System.dll!System.Net.HttpWebRequest.GetRequestStream(out System.Net.TransportContext context = null) + 0x1d3 bytes	
     	System.dll!System.Net.HttpWebRequest.GetRequestStream() + 0xe bytes	

    .NET Reflector on LazyAsyncResult.WaitForCompletion() shows that WaitOne() is called with an infinite timeout.

    This stack (and some other logs I have) shows that this occurs around a SSL/TLS handshake with the server.

     

     

    Context : .NET 3.5 SP1.

    On server side, HTTPS is handled by Apache, with a quite standard configuration.

     

    I can't find anything on the net about that. Is it a known bug ? Is there a workaround ?

     

    Thanks

    • Edited by Mathieu_D Thursday, February 10, 2011 2:21 PM improved description
    Thursday, February 10, 2011 11:12 AM

Answers

  • Appears to be KB980817

    As of now, I cannot confim the proposed hotfix corrects the problem (still testing).

     

    • Marked as answer by Mathieu_D Thursday, February 17, 2011 3:05 PM
    Thursday, February 17, 2011 3:02 PM

All replies

  • Appears to be KB980817

    As of now, I cannot confim the proposed hotfix corrects the problem (still testing).

     

    • Marked as answer by Mathieu_D Thursday, February 17, 2011 3:05 PM
    Thursday, February 17, 2011 3:02 PM
  • Hi,

    I'm stuck on the same problem, the hotfix was installed but nothing changed.

    Is it still relevant? Does it fix the problem?

    HELP!

    Idan.

    Wednesday, March 9, 2011 2:32 PM
  • I have a same problem(Windows 7(x64),Visual Studio 2010,.NET 4.0). When I use the HttpWebRequest.GetRequestStream() API,the main thread hangs and waits about 10-20 seconds(always random time).
    I tryed to use KB 980817, but have same problem.
    Is this bug already fixed?
    Tuesday, June 7, 2011 8:45 AM
  • you'd make a good systems tester for msft.... stack folks.
    Javaman, Cowboy Coders Unite!
    Thursday, June 9, 2011 2:44 AM
  • I am experiencing the same or a similar problem.  In this case, I have also written the server code which is listening for requests using HttpListener.  The problem occurs both when server and client are running in different processes in the same machine, and when they're in different machines.  I am targetting .Net v2.0 for both client and server, and using straight http, not https.

    The exchange will run for a random amount of time, but then simply stop.  The client is halted at the GetRequestStream call, and the server is sitting at the HttpListener.GetContext() call.  I have traces on either side of the server call, so I know that the server was waiting for the request, but did not receive it.  I have checked the request parameters and they seem normal - the same as all the preceding requests which used the same code.

    The client thread stack is as follows.

     

    [In a sleep, wait, or join]  

    mscorlib.dll!System.Threading.WaitHandle.WaitOne(timeout, exitContext)  

    mscorlib.dll!System.Threading.WaitHandle.WaitOne(millisecondsTimeout, exitContext)  

    System.dll!System.Net.LazyAsyncResult.WaitForCompletion(snap = true)  

    System.dll!System.Net.LazyAsyncResult.InternalWaitForCompletion()  

    System.dll!System.Net.Connection.SubmitRequest(request = {System.Net.HttpWebRequest})  

    System.dll!System.Net.ServicePoint.SubmitRequest(request = {System.Net.HttpWebRequest}, connName = "S>")  

    System.dll!System.Net.HttpWebRequest.SubmitRequest(servicePoint = {System.Net.ServicePoint})  

    System.dll!System.Net.HttpWebRequest.BeginSubmitRequest()  

    System.dll!System.Net.HttpWebRequest.GetRequestStream(context = null)  

    System.dll!System.Net.HttpWebRequest.GetRequestStream()  

    Does this provide any more clues?  Hope so!!
    Regards,       Stuart

     

    Thursday, June 9, 2011 10:24 AM
  • Has there been a resolution to this problem. It is happenning with me constantly.
    Elie
    Monday, June 27, 2011 6:48 PM
  • Hi all,

     

    In fact I did find a resolution to this problem - apologies for not posting it before.  I don't know whether this will also solve your problems also, but for what it's worth...

    I found that there seems to be a race condition inside HttpWebRequest or one of its base classes.  If you have >1 thread creating these objects for the same URL concurrently then the problem can occur.  This happens whether you use GetRequestStream or BeginGetRequestStream.

    The simple solution is to put a lock round the offending section, e.g.

     

    	lock (getRequestStreamSync) {
    		webRequest = (HttpWebRequest)WebRequest.Create(Uri);
    		webRequest.Method = "POST";
    		webRequest.Timeout = timeout;
    		webRequest.ContentType = "application/octet-stream";
    		webRequest.ContentLength = message.Length;
    
    		// start waiting for a request stream
    		sync.Reset();
    		object[] asyncState = new object[] { webRequest, sync };
    		webRequest.BeginGetRequestStream(WriteStreamCallBack, asyncState);
    		object result = sync.Wait(5000);
    		if (sync.TimedOut)
    			throw new Exception("Timed out");
    		if (result is Exception)
    			throw new TargetInvocationException((Exception)result);
    		stream = (Stream)result;
    	}
    
    

     

    This doesn't show the code that writes the posted data, but it is simple and does as you would expect, finishing with a sync.Signal. (sync is a simple synchronisation class.)  This immediately and completely solved the problem for me.  Hope it works for you!

    Just to clarify - the WriteStreamCallBack method doesn't write the data, just gets the stream and hands it back, so the lock is not too intrusive.

     

    Regards,      Stuart F.


    Monday, June 27, 2011 7:09 PM
  • request.GetRequestStream.Dispose()
    response.GetResponseStream.Dispose()
    Saturday, October 6, 2012 10:19 PM
  • We have exactly the same problem, but with simple HTTP.   A WinForms application connects to web services (.net 2.0 on both sides) using a wrapper over SoapHttpClientProtocol class.

    When we call SoapHttpClientProtocol .Invoke , it calls HttpWebRequest.GetRequestStream().  We have multiple threads which call webmethods at once, and each of them has its own instance of HttpWebRequest (we overrided GetWebRequest method in our wrapper class derived from SoapHttpClientProtocol).

    In this method we replace the URL by adding ?R=93843294834 (random number) at the end, so the call to asmx works fine (server part doesn't care about query string), but the client part has all requests Urls different.

    However, this doesn't help  - HttpWebRequest.GetRequestStream() hangs from time to time. For example, we have a form where 4 requests are sent to server simulteneously,  the 5th request is hit when we click on a tab in a window. And this hangs.... We attach MDbg and see where the main thread is. It hangs in that two WaitOne() calls - exactly as shown above.

    Since we are confident that HttpWebRequest objects are different, the conclusion is next:  there is some annoying static field in  HttpWebRequest class and some bugs which doesn't allow multiple requests to come out.

    Currently I see only one solution: to reflect System.Web.Services,  to fix the bugs and recompile it completely.  I see no other ways out.

    OR, to find an alternative to HttpWebRequest implementation (maybe some open source - whatever). 

    Does anyone see a better way out?

    Tuesday, December 18, 2012 8:04 PM
  • I have found a way out at

    http://stackoverflow.com/questions/1361771/max-number-of-concurrent-httpwebrequests

    Do the following:

    System.Net.ServicePointManager.DefaultConnectionLimit = 1600;

    set  this somewhere in static constructor BEFORE any request will go out. This really helps.

    • Proposed as answer by Ihor Bobak Wednesday, December 19, 2012 1:10 PM
    Wednesday, December 19, 2012 1:10 PM