Reconfigure net.tcp WCF binding for Azure load balancing

Question

We are receiving a variety of errors when our net.tcp connections are load balanced in the Azure cloud. The errors include:

• An unsecured or incorrectly secured fault was received from the other party
• The communication object, System.ServiceModel.Channels.ServiceChannel, cannot be used for communication because it has been Aborted
• The communication object, System.ServiceModel.Channels.SslStreamSecurityUpgradeProvider, cannot be used for communication because it has been Aborted.

The errors occur when our client application performs many concurrent requests. The number of requests increases the number of connections to the service and when any of them target a different server it fails.

I understand that we need to turn off sessions, or at least use stateful session tokens, but I'm at a loss as to how to do this. I've tried setting requireSecurityContextCancellation to false and this works in the DevFabric but does not seem to work in the cloud. Perhaps because the cloud service does not run under an account with a user profile.

I've listed the binding we are using below. Sorry it is in so much detail, but we have had to tweak a number of settings to get things running reliably on-premise. We also use a second binding with the same settings except "transferMode=Streamed".

Can anyone please let me know what I need to change in this binding to get it to work reliably behind the Azure load balancers?

<binding name="EncryptedUserNameBinding" 
         sendTimeout="00:10:00" 
         receiveTimeout="00:10:00">
     <transactionFlow transactionProtocol="OleTransactions" />
     <security defaultAlgorithmSuite="Default" 
          authenticationMode="SecureConversation" 
          requireDerivedKeys="false" 
          securityHeaderLayout="Strict" 
          includeTimestamp="true" 
          keyEntropyMode="CombinedEntropy" 
          messageSecurityVersion="Default" 
          requireSecurityContextCancellation="true">
      <localClientSettings cacheCookies="true" 
                 detectReplays="false" 
                 replayCacheSize="900000" 
                 maxClockSkew="01:00:00"
                 maxCookieCachingTime="Infinite" 
                 replayWindow="00:05:00" 
                 sessionKeyRenewalInterval="10:00:00" 
                 sessionKeyRolloverInterval="00:05:00" 
                 reconnectTransportOnFailure="false" 
                 timestampValidityDuration="00:05:00" 
                 cookieRenewalThresholdPercentage="60" />
      <localServiceSettings detectReplays="false" 
                 issuedCookieLifetime="10:00:00" 
                 maxStatefulNegotiations="128" 
                 replayCacheSize="900000"
                 maxClockSkew="01:00:00" 
                 negotiationTimeout="00:01:00" 
                 replayWindow="00:05:00" 
                 inactivityTimeout="00:02:00" 
                 sessionKeyRenewalInterval="15:00:00"
                 sessionKeyRolloverInterval="00:05:00" 
                 reconnectTransportOnFailure="false"
                 maxPendingSessions="128"
                 maxCachedCookies="5000" 
                 timestampValidityDuration="00:05:00" />
      <secureConversationBootstrap defaultAlgorithmSuite="Default" 
                     authenticationMode="UserNameOverTransport" 
                     requireDerivedKeys="true" 
                     securityHeaderLayout="Strict" 
                     includeTimestamp="true" 
                     keyEntropyMode="CombinedEntropy" 
                     messageSecurityVersion="Default">
       <localClientSettings cacheCookies="true" 
                  detectReplays="false" 
                  replayCacheSize="900000" 
                  maxClockSkew="01:00:00" 
                  maxCookieCachingTime="Infinite" 
                  replayWindow="00:05:00" 
                  sessionKeyRenewalInterval="10:00:00" 
                  sessionKeyRolloverInterval="00:05:00" 
                  reconnectTransportOnFailure="false" 
                  timestampValidityDuration="00:05:00" 
                  cookieRenewalThresholdPercentage="60" />
       <localServiceSettings detectReplays="false" 
                  issuedCookieLifetime="00:15:00" 
                  maxStatefulNegotiations="128" 
                  replayCacheSize="900000" 
                  maxClockSkew="01:00:00" 
                  negotiationTimeout="00:01:00" 
                  replayWindow="00:05:00" 
                  inactivityTimeout="00:02:00" 
                  sessionKeyRenewalInterval="15:00:00" 
                  sessionKeyRolloverInterval="00:05:00" 
                  reconnectTransportOnFailure="false"
                  maxPendingSessions="128" 
                  maxCachedCookies="1000" 
                  timestampValidityDuration="00:05:00" />
      </secureConversationBootstrap>
     </security>
     <binaryMessageEncoding maxReadPoolSize="64" 
                 maxWritePoolSize="16" 
                 maxSessionSize="2048">
      <readerQuotas maxDepth="32"
             maxStringContentLength="32768"
             maxArrayLength="16384" 
             maxBytesPerRead="16384"
             maxNameTableCharCount="16384" />
     </binaryMessageEncoding>
     <sslStreamSecurity requireClientCertificate="false" />
     <tcpTransport manualAddressing="false" 
            maxBufferPoolSize="524288" 
            maxReceivedMessageSize="5000000" 
            connectionBufferSize="8192" 
            hostNameComparisonMode="StrongWildcard"
            channelInitializationTimeout="00:00:05"
            maxBufferSize="5000000"
            maxPendingConnections="10"
            maxOutputDelay="00:00:00.2000000" 
            maxPendingAccepts="10" 
            transferMode="Buffered" 
            listenBacklog="10" 
            portSharingEnabled="false" 
            teredoEnabled="false">
      <connectionPoolSettings groupName="default" 
                  leaseTimeout="00:05:00" 
                  idleTimeout="00:10:00"
                  maxOutboundConnectionsPerEndpoint="5000" />
     </tcpTransport>
    </binding>

Answer 1

Hi Glenn,

Are you hosting the WCF service in a worker role?

Please refer to the code sample WCF Azure Net.TCP Keep Alive.

Answer 2

I'm hosting in a worker role.

I don't believe this is an issue related to connections being closed, therefore keep-alive will not help.

The observations I've made show that the original connection is still active and when a number of requests are made from the client concurrently, the number of connections increases. It is at this point that the calls can fail.