locked
Multiple Concurrent HttpWebRequests RRS feed

  • Question

  • I am stress testing a web site to determine the effects of multiple concurrent web requests.  To simulate this I have created a WinApp that creates a desired number of background threads.  Each thread initiates a while loop that creates an HttpWebRequest, sets its headers as:

    httpRequest.Method = "POST";  
    httpRequest.ContentType = "application/x-www-form-urlencoded";  
    httpRequest.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 2.0.50727; .NET CLR 1.1.4322; .NET CLR 3.0.04506.30)";  
    httpRequest.Timeout = 110000;  
    httpRequest.KeepAlive = true;  
    httpRequest.SendChunked = false;  
    httpRequest.ContentLength = strMsg.Length;  
    httpRequest.ProtocolVersion = HttpVersion.Version11;  
    httpRequest.Headers.Add("Accept-Language""en-us");  
    httpRequest.Headers.Add("Accept-Encoding""gzip, deflate");  
    httpRequest.Headers.Add("Pragma""no-cache"); 

    I then call .GetResponse() which will block until a repsonse is sent by the website.  The website is configured to hold the request until data is available to send or 100000 ms has passed.  With no data each request should be held for 100 seconds then an empty response sent back to the WinApp.  This all seems to be working well, except that when I view the number of concurrent requests executing I am only seeing 2 or so when there should be 100.  The website is on iis6 and is configured as a multi-worker process web garden.  So with at least two wp and the maxWorkerThread set to 100, I should be able to see a max of 184 concurrent requests. 

    I have found that if I run Fiddler2 while testing, the number of concurrent requests does reach the desired count.  As soon as I close Fiddler2 the request count slowly declines back to 2, as requests are responded to after the set amount of time.

    Does someone have an explanation for this and a resolution for me to work without needing Fiddler2?

    Thanks.

    Tuesday, January 20, 2009 4:46 PM

Answers

  • The "problem" is httpRequest.ServicePoint.ConnectionLimit, which defaults to 2 for remote hosts (the RFC-recommended maximum number of concurrent connections from a client to a single server).  You can bump up this limit to allow more concurrent connections for a load-test client.

    It's working with Fiddler in the middle because Fiddler works by registering itself as a proxy server running on the local machine - programs which are proxy-aware send requests to Fiddler instead of directly to the final destination, Fiddler logs them, and then passes them on to the final destination.  When HttpWebRequest thinks its talking to the local machine it doesn't limit the number of concurrent connections, so more concurrent connections are made.

    -dave
    Thursday, January 22, 2009 1:54 AM
    Moderator