Only one usage of each socket address (protocol/network address/port) is normally permitted, web service issue
I have several application that connect to a .asmx web service on the intranet which is consumed by a link such as the following: http://MyWebService.mycompany.com:3456/MyService.asmx
Now very rarely, and without pattern or specific reasoning (I mean once every month; maybe) I will receive the following error:
System.Net.Sockets.SocketException: Only one usage of each socket address (protocol/network address/port) is normally permitted 220.127.116.11:3428 at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress) at System.Net.Sockets.Socket.InternalConnect(EndPoint remoteEP) at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Int32 timeout, Exception& exception)
Now all of the research I have done, has been in regards to developers with a similar issue, but there issue arises in making socket calls or TCP/IP calls, not with using web services.
I also read this may have to do with TCP/IP port exhaution but I couldn;t realte that back to a .asmx web service that communicates via http. So I need a little help understanding the root cause of why this issue occurs, and how I could possibly modify my code to prevent this exception. Just to give a littel except of code and how this service is called, take a look to the following below:
'Instatiate and set up credentials to authenticate to the web service Dim ws As New MyService Try 'Set web service credentials wsDAL.Credentials = New NetworkCredential(UserName, Password, DomainName) wsDAL.PreAuthenticate = True 'Call web service ws.DoSomethingImportant() Finally 'Clean Up ws.Dispose() End TryAny help is appreciated, thanks!Wednesday, January 06, 2010 2:28 PM
You consume the web service via TCP and this is a TCP error, so stop looking at "web service" and start looking at TCP issue. The error message - does it report your local address and port as being busy? If yes, then this means that your computer ran out of ports. use TCPView utility to see what ports (and how many) are open and what application doesn't release the ports right. This can be .NET issue, i.e. it doesn't close some socket. If this is your application that doesn't close sockets properly, then you need to review your code and then search for information what class you use for accessing the web service and whether that class closes the sockets right or you need to call some additional method to close the socket.Thursday, January 07, 2010 1:21 PM
Yes I agree this is a TCP issue and not a web service specefic issue. I just mentioned it to give a full picture of the scenario to help with suggestions.
You see the code I am using. Do you have any specefic code examples to provide to help ensure the socket is closed when calling the web service?
Remember with the web service the proxy is generated and deals with the direct calls; I can modify the proxy if needed to check to make sure the port is either/open and gets closed correctly. Other than that, I am just calling .Dispose() on the web service when finished; that is it.
One issue I have is this occurs so rarily and unexpected (like once every few months), so trying to use a utility to monitor the situation when it happens is almost impossible. I feel I need to tighten up the code to not make the call if there is no port avaliable, or making sure all calls do a better job of releasing the port when finished. Any code to that end will help if you can provide it, or point me in the right direction.
Thank you,Monday, January 11, 2010 4:59 PM
You might find this link of interest:
concerning ephimeral port exhaustion. Even if socket is correctly closed, under heavy load you might experience the problem due to the long TIME_WAIT delay.
Hope it helps
Tuesday, February 02, 2010 9:45 AM
Thank you for the response. I have had my eye on this thread and hoping for some more responses. Yes, interestingly enough I read that exact article some time back, but the MSDN bread crumb trail indicated it was for 'BizTalk' servers and communication. However, I agree that some of the solutions like modifying the 'TIME_WAIT' value in the registry seems to be a more generic solution, and possibly something to try.
However, honestly I would much rather take a programatic approach to solving this solution. If I go modifying the registry values on our production servers to fix one thing, but break another, I will not be any better off.
Ideally, I wanted to know if there was a programatic way when making the service call to either check the state of the socket and only call if it is free. Do you or anyone have any ideas on this possibly?Tuesday, February 02, 2010 4:05 PM
I do agree that tweaking a production server like this must be done with a very thoughtful mind, so I discourage such approach unless you have no other option (and a good backup).
I had the exact same problem on a PBX VoIP computer, with an application receiving notifications and issuing commands through http requests (HttpListener and WebClient classes). Considering the heavy load of the call-center (hundreds of clients simultaneously) and high number of requests per client per second, no big wonder I faced the port exhaustion problem.
I solved it with all 3 approaches:
- programmatically I reduced as much as possible, latency, waiting time and overheads: lowered Timeout properties for connections, issued httprequests with PreAuthenticate set to true, used ThreadPool'd connections.
- Raised the number of available ports from 5000 to 10000. Could have done more, but didn't want to impact too much on the kernel's need for resources.
- Lowered the delay of the TIME_WAIT down to 30s.
I don't claim this is the ultimate solution for everyone, and I got there with some trial&error procedure, which is not suitable for a mission-critical enviroment, but it solved my problem pretty smoothly.
Depending on your set-up, I recommend an analysis of the usage of http resources in code, and some profiling (I know specific tools exist, but I have no direct knowledge nor experience on them) of the exact rate of their usage in time.
Also a network analysis with Wireshark-like tools could contribute in tracking down the real source of the problem.
Hope these two cents of mine help you out.
Tuesday, February 02, 2010 4:44 PM
One question on the following from your post:
"Raised the number of available ports from 5000 to 10000. Could have done more, but didn't want to impact too much on the kernel's need for resources."
What happens in my case where the port is not arbitrary; meaning within my consumed service reference and how the .asmx service is set up it is working on a specefic port like shown below:
So does increasing the number of ports still pertain to a connection like this?
Thank you,Tuesday, February 02, 2010 8:27 PM