Problems when restarting a node

Question

I use Windows Server AppFabric, Cache Cluster contains two nodes on Windows Server 2008 R2 (server1 and server2), Cluster Configuration Storage Location is SQL Server.

Problems:

1. When I disable SERVER1, then SERVER2 after some time stops working (about 5 minutes). does not help even Restart-CacheCluster. When i disable SERVER2, SERVER1 works.

2. Task Scheduler contains  task with trigger on “At Startup”,  

import-module DistributedCacheAdministration   
$computer = gc env:computername 
use-cachecluster 
start-cachehost $computer 22233

At SERVER2 script works off without error, on SERVER1 an error occurs. Here he writes to the log

76c5a26000000000000000000000000 failed to refresh lookup table with exception Microsoft.Fabric.Common.OperationCompletedException: Operation completed with an exception ---> Microsoft.Fabric.Federation.MessageDroppedByDestinationException: Message is dropped by the destination
--- End of inner exception stack trace ---
at Microsoft.Fabric.Common.OperationContext.End()
at Microsoft.Fabric.Federation.FederationSite.EndRoutedSendReceive(IAsyncResult ar)
at Microsoft.Fabric.Data.ReliableServiceManager.EndRefreshLookupTable(IAsyncResult ar),CASSend.RefreshLookupTable.76c5a26000000000000000000000000,Warning,2012-3-27 15:52:23.713

Any suggestions? Thanks

Answer 1

The SCM error may suggest the powershell window's user is not an admin on server2, the person administering the cluster needs to be admin on all cache clusters otherwise it wont be able to do functions. Try to store the exception in a powershell object and dump it.

Also, are you using onloading ? If yes, the behaviour is expected, please offload cluster management for a 2 node cluster.

Can you share your cluster config  ?

Also consider running the BPA tool :

http://www.microsoft.com/en-us/download/details.aspx?id=29051

Note that the BPA tools requires remote powershell to be on and the user running it to have appropriate privileges on the cluster servers.

Answer 2

About onloading, i dont know. By default.

My config

<?xml version="1.0" encoding="utf-8"?>
<configuration>
    <configSections>
        <section name="dataCache" type="Microsoft.ApplicationServer.Caching.DataCacheSection, Microsoft.ApplicationServer.Caching.Core, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
    </configSections>
    <dataCache size="Small">
        <caches>
            <cache consistency="StrongConsistency" name="Site1">
                <policy>
                    <eviction type="Lru" />
                    <expiration defaultTTL="10" isExpirable="true" />
                    <serverNotification isEnabled="true" />
                </policy>
            </cache>
            <cache consistency="StrongConsistency" name="Site2">
                <policy>
                    <eviction type="Lru" />
                    <expiration defaultTTL="10" isExpirable="true" />
                    <serverNotification isEnabled="true" />
                </policy>
            </cache>
            <cache consistency="StrongConsistency" name="default">
                <policy>
                    <eviction type="Lru" />
                    <expiration defaultTTL="10" isExpirable="true" />
                </policy>
            </cache>
        </caches>
        <hosts>
            <host replicationPort="22236" arbitrationPort="22235" clusterPort="22234"
                hostId="1397468738" size="8191" leadHost="true" account="dev\Server1$"
                cacheHostName="AppFabricCachingService" name="Server1" cachePort="22233" />
            <host replicationPort="22236" arbitrationPort="22235" clusterPort="22234"
                hostId="1101426491" size="8191" leadHost="false" account="dev\Server2$"
                cacheHostName="AppFabricCachingService" name="Server2" cachePort="22233" />
        </hosts>
        <advancedProperties>
            <securityProperties mode="None" protectionLevel="None" />
        </advancedProperties>
    </dataCache>
</configuration>

BPA tool was running, no errors detected.

Thanks

Answer 3

add this to the advancedproperties section

<partitionStoreConnectionSettings leadHostManagement="false" />

then

set  leadHost="false" on Server1

otherwise when server1 is stopped it will stop the whole cluster as at least two nodes are required to be running if

leadhostmanagment is set to "true"

checkout thsi for more info

http://msdn.microsoft.com/en-us/library/ee790895.aspx