locked
Bizarre issue with ASP.Net web site migrating to new web server RRS feed

  • Question

  • User1128408901 posted

    We have an ASP.Net web site (v3.5 framework) that is running on IIS 7.5/Server2008R2. I'm testing this site on a new web server running IIS 8.5/Server2012R2.

    Randomly, it seems to just lose connectivity to our SQL server, message is this:

    Connection not valid. A network-related or instance-specific error occurred while establishing a connection to
    SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that 
    SQL Server is configured to allow remote connections. (provider: TCP Provider, error: 0 - A connection attempt 
    failed because the connected party did not properly respond after a period of time, or established connection 
    failed because connected host has failed to respond.)
    

    Occasionally, we also get the standard "SQL Server does not exist or access is denied" message like the SQL server is disconnected or down. Also, occasionally, the message about "...could not obtain a connection from the pool." The SQL server is definitely NOT down or disconnected as it's the same server and database as the current site running on 2008R2, which experiences no issues.

    The weird thing is, once I get that message, my session appears to be "ruined" and all pages return this error for my session only, while others can still browse the site OK. If I close out all my browser windows and pull the site back up, everything works fine until (randomly, again) the message appears, then it's broken again. Recycling the app pool also fixes it until, again, I get the message once, then all subsequent attempts fail.

    Things I've tried so far:

    • Changed all connection strings to use IP address and port number instead of machine name to rule out DNS or other name resolution issues
    • Disabled Windows firewall on both SQL server and web server
    • Added "Pooling=False" to connection strings to disable pooling
    • Added "Max Pool Size=10000" to make sure the connection pool was ridiculously large
    • Checked and double-checked every application pool and web site setting in IIS that I can think of to make sure they are the same between old and new server

    Any other ideas? I'm totally at a loss. I have managed to identify a sequence of events that I can repeat to get the error to happen every time. This exact same code base is running on the existing 2008R2 server just fine, checked all the app pool settings and they are the same.

    Any differences between IIS 7.5 and 8.5 that might be causing this?

    Tuesday, July 18, 2017 11:09 PM

All replies

  • User1771544211 posted

    Hi abgibbs,

    Any differences between IIS 7.5 and 8.5 that might be causing this?

    Based on my experience, there's any difference between IIS 7.5 and 8.5 can cause your issue. The following link shows the what's new in IIS 8.5, please take it as reference.

    https://docs.microsoft.com/en-us/iis/get-started/whats-new-in-iis-85/enhanced-logging-for-iis85

    Occasionally, we also get the standard "SQL Server does not exist or access is denied" message like the SQL server is disconnected or down.

    Since this issue is occasionally happening, the connection string should be correct. Perhaps this issue is related to the network, if the network breaks occasionally, it may cause this issue. 

    Is the SQL Server in the same server as the IIS Server? If not, please check if the network connection is stable.

    The weird thing is, once I get that message, my session appears to be "ruined" and all pages return this error for my session only, while others can still browse the site OK.

    That's really strange that only our session is "ruined" and other session can still work. I can't think of a condition that can cause this issue.

    Best Regards,

    Jean

    Wednesday, July 19, 2017 6:31 AM
  • User475983607 posted

    abgibbs

    Occasionally, we also get the standard "SQL Server does not exist or access is denied" message like the SQL server is disconnected or down. Also, occasionally, the message about "...could not obtain a connection from the pool." The SQL server is definitely NOT down or disconnected as it's the same server and database as the current site running on 2008R2, which experiences no issues.

    You probably have bugs in the code base where connections are not properly closed, an open transaction, or a lock.  The issue does not show until there is enough load or the users hit the pages that have the bug.

    Wednesday, July 19, 2017 10:26 AM
  • User753101303 posted

    Hi,

    If you really "managed to identify a sequence of events" have a closer look at the involved code? Do you try to store a connection in your session or something like that ?  Or do you use static variables?

    "...could not obtain a connection from the pool" would point to not properly releasing connection. To be on the safe side it's likely best to quote the exact error message.

    Could it be : "The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached."

    I would really fix this (rather than trying to disable pooling or increasing the pool size) in case it would play a role in your current issue.

    Wednesday, July 19, 2017 10:46 AM
  • User1128408901 posted

    Some more info to answer a few questions:

    I'm not the only person working on this site, but we are generally very good about making sure all DB connections are closed.  We have a helper class we use for all SQL operations and it closes out connections automatically.  We also tend to stay away from using session variables as much as possible since they are so easily abused, and this is a fairly heavily trafficked site with around 20,000 users daily.  However, as I said multiple people are working on this site, and I'm not standing over the shoulder of every developer, and not always in the habit of reading every changeset checked into TFS, so I'll admit there is a small possibility there could be a coding issue.

    What's driving me nuts is, this **EXACT** same code works fine on a 2008R2 server running IIS 7.5.  I even tried multiple 2008R2 servers and we have no issues at all, under load or not.  I'm using TFS to deploy the code, it's a single release definition with multiple environments, one is a 2008R2 server the other running 2012R2.  The methods I use to duplicate the error on 2012R2, work perfectly fine on 2008R2.

    Wednesday, July 19, 2017 4:20 PM
  • User753101303 posted

    The problem is that you are really half way beetween an application error and a truly instable network connection.

    What I find strange is "The weird thing is, once I get that message, my session appears to be "ruined" and all pages return this error for my session only, while others can still browse the site OK." If you meant the SQL Server connectivity message it seems to point at having the connection being tied to your "session" (browser session or  maybe authentication especially if Windows auth is used).
    Also that you DO have a way to reproduce the error for sure. So it would create some kind of pattern that make the network to fail (and for you only ?) Ah, do you use integrated security and maybe impersonation ? Antivirus ?

    Apart the fact that the error seems to be tied to a particular user it looks like an intermittent network connectivity loss (my understanding is that the connection request is really sent but no response is ever received).

    Edit: try perhaps a SQL Server forum at https://social.msdn.microsoft.com/Forums/en-US/home. You might have better support there if it turns out to be a network level issue.

    Edit2 : from the know sequence of events that make the connection to fail and if not too hard I would try perhaps to create a test page that would replay the same queries  to see what happens???

    Wednesday, July 19, 2017 5:31 PM