TFS server connection ____

Question

We are running TFS 2008 on one server and SQL Server 2005 on a separate machine. We have approx. 90 developers using the system. Contact is made via both VSTS 2008 and the web.

For the last couple of months, the users have been reporting intermittent connection issues. Sometimes, users report errors similar to the following:

Microsoft.TeamFoundation.VersionControl.Client.VersionControlException: TF30041: Team Foundation Server could not connect to the database. Contact your Team Foundation Server administrator. ---> System.Web.Services.Protocols.SoapException: TF30041: Team Foundation Server could not connect to the database. Contact your Team Foundation Server administrator.

and

Microsoft.TeamFoundation.VersionControl.Client.RepositoryNotFoundException: Team Foundation Server http://cstfs01.premconf.com:8080 does not exist or is not accessible at this time. Technical information (for administrator): The underlying connection was closed: A connection that was expected to be kept alive was closed by the server. ---> System.Net.WebException: The underlying connection was closed: A connection that was expected to be kept alive was closed by the server. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host at System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)

and

Microsoft.TeamFoundation.VersionControl.Client.VersionControlException: TF10217: Team Foundation was unable to complete your request due to an unexpected error. For more details, see the event log on the Team Foundation Server. ---> System.Web.Services.Protocols.SoapException: TF10217: Team Foundation was unable to complete your request due to an unexpected error. For more details, see the event log on the Team Foundation Server.

and

Microsoft.TeamFoundation.VersionControl.Client.RepositoryNotFoundException: Team Foundation Server http://cstfs01.premconf.com:8080 does not exist or is not accessible at this time. Technical information (for administrator): Unable to connect to the remote server ---> System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 10.3.2.44:8080

and

Microsoft.TeamFoundation.VersionControl.Client.RepositoryNotFoundException: Team Foundation Server http://cstfs01.premconf.com:8080 does not exist or is not accessible at this time. Technical information (for administrator): The underlying connection was closed: An unexpected error occurred on a receive. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host

In MSDN, I found the note re. disabling keep-alives. I've done that, and some of the complaints have cleared up.

However, we still have issues wherein connection failures occur intermittently. Different users are affected at different times. Looking at the servers, I can find no obvious bottlenecks, nor can I find any obvious clues in the logs. Indeed, during episodes where the lockups are occurring, I can find no evidence that the server is under any significant load.  The symptoms are:

1) VSTS locks up while a connection to the server is attempted, and

2) Sometimes the lockup clears; sometimes the user has to restart VSTS; sometimes the users has to reboot.

This can affect one user, or two or three, or a dozen.

If anyone has any thoughts, I'd really appreciate hearing them. Thanks.

Answer 1

I have 2 thoughts for you:

1) When you have a virtualized AT, then connections may be dropped when the hardware resources on the host are stressed

2) When you are using TSWA, there is a memory leak in TSWA. You have to recycle the app pool. This might cause the closing connections.

---

Ewald - Please remember to mark the replies as answers if they help.


Blog: www.ewaldhofman.nl

Answer 2

Hello,

Do you see any of these errors in the TFS server logs? Most of these errors (except the 1st one) suggest some sort of network problems. So, I would check the network configuration and the firewalls first (a faulty router or network card could cause these).

Then, I would look at the IIS app pool restarts - it is possible there is an underlying issue that's causing IIS to restart and cause these errors.



Thanks,
Alin,
PRAKTIK Consulting
TFS Hosting and TFS Consulting Services.

Answer 3

Continued research of the issue uncovered this: http://social.msdn.microsoft.com/Forums/en/tfswebaccess/thread/d0182cd4-c7a1-4710-a394-934430c5041d.

This information, plus info about TSWA and data-tier access on this page: http://msdn.microsoft.com/en-us/library/bb668952.aspx indicate that the memory leak is most likely our culprit.

Thank you, Ewald!