none
Failed to add node to cluster RRS feed

  • Question

  • Hello.  I can't add a note to newly created cluster.  It timeout while waiting for a response from the node being added.  I've gone through a lot of posts regarding this topic but none has resolved my issue.

    Failover cluster is currently configured on Guest A and I'm trying to add Guest B to the cluster.

    Host: Win2008R2 Hyper-V Core, Firewall disabled, SEP AV
    Guests: Win2008 R2 Ent, Firewall disabled, no AV
    User account: Domain and local admin

    I've tried the steps listed here http://sqlserverscribbles.com/2013/06/17/windows-cluster-freezes-at-waiting-for-notification-that-node-is-a-fully-functional-member-of-the-cluster/

    Only exception is I haven't disabled SEP on the Host Hyper-V but based on what I've read it shouldn't have any impact because the virtual network is supposed to act like it's on a switch.

    ----------------------------------------------------------

    6:17:37.565 INFO  [DCM] Processing message dcm/connectivityCheck
    6:17:37.565 INFO  [DCM] ConnectivityCheck optimize = false
    6:17:37.565 INFO  [DCM] NET 5ca4d5ed-a7f4-412c-a1f3-ac28a04266e7 metric 10000
    6:17:37.565 INFO  [DCM]     node 1 interface <obj sig='ITFC' id='20472c3b-c21a-4e07-83a6-ffd6fd20dd2d' name='SGCTRSQ004 - Local Area Connection'/>node Id:'1' adapterName:'Microsoft Virtual Machine Bus Network Adapter' addrPrefixSet:'<set size='1'>
    6:17:37.565 INFO      <item>192.168.1.146/24</item>
    6:17:37.565 INFO  </set>
    6:17:37.565 INFO  '
    6:17:37.565 INFO  [IM] State calculation complete without issuing pings
    6:17:37.565 INFO  [IM] Resetting interface state calculation state
    6:17:37.581 DBG   [NETFTAPI] received NsiParameterNotification  for ef80::59g3:29d1:ff68:6996 (IpDadStatePreferred )
    6:17:37.581 DBG   [NETFTAPI] received NsiParameterNotification  for 169.254.1.211 (IpDadStatePreferred )
    6:17:37.581 INFO  [NM] Received request from client address ::1.
    6:17:37.581 DBG   [NETFTAPI] Signaled NetftRemoteReachable  event, local address 192.168.1.146:003853 remote address 192.168.1.145:003853
    6:17:37.581 INFO  [FTI][Initiator] Got remote route reachable from netft evm. Setting state to Up for route from 192.168.1.146:~3343~ to 192.168.1.145:~3343~.
    6:17:37.581 INFO  [IM] got event: Remote endpoint 192.168.1.145:~3343~ reachable from 192.168.1.146:~3343~
    6:17:37.581 INFO  [IM] Route from 192.168.1.146:~3343~ to 192.168.1.145:~3343~ is already up, not sending report
    6:17:37.581 INFO  [TM] got event: Remote endpoint 192.168.1.145:~3343~ reachable from 192.168.1.146:~3343~
    6:17:37.581 INFO  [FTW] NetFT is ready after 0 msecs wait.
    6:17:37.581 INFO  [FTI][Initiator] Route is up and NetFT is ready. Connecting to node sgctrsq003 on virtual IP fe80::5c1a:c21e:713c:607c%19:~3343~
    6:17:38.708 DBG   [NETFTAPI] received NsiParameterNotification  for ef80::59g3:29d1:ff68:6996 (IpDadStatePreferred )
    6:17:40.712 DBG   [NETFTAPI] received NsiParameterNotification  for 169.254.1.211 (IpDadStatePreferred )
    6:17:58.622 INFO  [NODE] Node 1: New join with n2: stage: 'Establish Connection over NetFT Adapter' status WSAETIMEDOUT(10060) reason: 'Failed to connect to remote endpoint fe80::5c1a:c21e:713c:607c%19:~3343~'
    6:17:58.622 INFO  [CORE] Node 1: Clearing cookie 02f18184-4755-459f-a275-3bd47b804bab
    6:18:38.606 INFO  [ACCEPT] 0.0.0.0:~3343~: Accepted inbound connection from remote endpoint 192.168.1.145:~51442~.
    6:18:38.606 INFO  [SV] Securing route from (192.168.1.146:~3343~) to remote  (192.168.1.145:~51442~).
    6:18:38.606 INFO  [SV] Got a new incoming stream from 192.168.1.145:~51442~
    6:18:38.622 INFO  [SV] Authentication and authorization were successful
    6:18:38.622 INFO  [SV] Security Handshake successful while obtaining SecurityContext for NetFT driver
    6:18:38.622 INFO  [VER] Got new TCP connection. Exchanging version data.
    6:18:38.622 INFO  [VER] Checking version compatibility for node sgctrsq003 id 2 with following versions: highest [Major 6 Minor 7601 Upgrade 7 ClusterVersion 0x00061DB1], lowest [Major 6 Minor 7601 Upgrade 7 ClusterVersion 0x00061DB1].
    6:18:38.622 INFO  [VER] Version check passed: node and cluster highest supported versions match.
    6:18:38.622 INFO  [SV] Negotiating message security level.
    6:18:38.622 INFO  [SV] Already protecting connection with message security level 'Sign'.
    6:18:38.622 INFO  [FTI] Got new raw TCP/IP connection.
    6:18:38.622 INFO  [FTI][Initiator] This node (1) is initiator
    6:18:38.622 INFO  [FTI][Initiator] Trying to select best endpoints among 169.254.1.211:~3343~, ef80::59g3:29d1:ff68:6996:~3343~ (first pair) and 169.254.2.217:~3343~, fe80::5c1a:c21e:713c:607c:~3343~ (second pair)
    6:18:38.622 INFO  [StreamDb] Route virtual ef80::59g3:29d1:ff68:6996:~0~ to virtual fe80::5c1a:c21e:713c:607c:~0~ added
    6:18:38.622 INFO  [NETFT] Route <struct mscs::FaultTolerantRoute>
    6:18:38.622 INFO    <realLocal>192.168.1.146:~3343~</realLocal>
    6:18:38.622 DBG   [NETFTAPI] Signaled NetftRemoteUnreachable  event, local address 192.168.1.146:003853 remote address 192.168.1.145:003853
    6:18:38.622 INFO    <realRemote>192.168.1.145:~3343~</realRemote>
    6:18:38.622 INFO    <virtualLocal>ef80::59g3:29d1:ff68:6996:~0~</virtualLocal>
    6:18:38.622 INFO    <virtualRemote>fe80::5c1a:c21e:713c:607c:~0~</virtualRemote>
    6:18:38.622 INFO    <Delay>1000</Delay>
    6:18:38.622 INFO    <Threshold>5</Threshold>
    6:18:38.622 INFO    <Priority>99</Priority>
    6:18:38.622 INFO    <Attributes>2147483649</Attributes>
    6:18:38.622 INFO  </struct mscs::FaultTolerantRoute>
    6:18:38.622 INFO   removed
    6:18:38.622 INFO  [IM] got event: Remote endpoint 192.168.1.145:~3343~ unreachable from 192.168.1.146:~3343~
    6:18:38.622 INFO  [IM] Marking Route from 192.168.1.146:~3343~ to 192.168.1.145:~3343~ as down
    6:18:38.622 INFO  [CORE] Node 1: Clearing cookie 02f18184-4755-459f-a275-3bd47b804bab
    6:18:38.622 INFO  [NDP] Checking to see if all routes for route (virtual) local ef80::59g3:29d1:ff68:6996:~0~ to remote fe80::5c1a:c21e:713c:607c:~0~ are down
    6:18:38.622 INFO  [NDP] All routes for route (virtual) local ef80::59g3:29d1:ff68:6996:~0~ to remote fe80::5c1a:c21e:713c:607c:~0~ are down
    6:18:38.622 WARN  mscs::ListenerWorker::operator (): HrError(0xd0000043)' because of '::NetftAddRoute( handle.handle, netFtRoute.get(), &netftSecurityContext )'
    6:18:38.622 INFO  [IM] Adding information for route Route from local 192.168.1.146:~0~ to remote 192.168.1.145:~0~, status: false, attributes: 0
    6:18:38.622 ERR   [IM] Unable to find adapter
    6:18:38.622 INFO  [IM] Sending connectivity report to leader (node 1): <class mscs::InterfaceReport>
    6:18:38.622 INFO    <fromInterface>20472c3b-c21a-4e07-83a6-ffd6fd20dd2d</fromInterface>
    6:18:38.622 INFO    <upInterfaces><vector len='1'>
    6:18:38.622 INFO      <item>20472c3b-c21a-4e07-83a6-ffd6fd20dd2d</item>
    6:18:38.622 INFO  </vector>
    6:18:38.622 INFO  </upInterfaces>
    6:18:38.622 INFO    <downInterfaces><vector len='0'>
    6:18:38.622 INFO  </vector>
    6:18:38.622 INFO  </downInterfaces>
    6:18:38.622 INFO    <viewId>001</viewId>
    6:18:38.622 INFO  </class mscs::InterfaceReport>
    6:18:38.622 INFO  [IM] Leader got report <class mscs::InterfaceReport>
    6:18:38.622 INFO    <fromInterface>20472c3b-c21a-4e07-83a6-ffd6fd20dd2d</fromInterface>
    6:18:38.622 INFO    <upInterfaces><vector len='1'>
    6:18:38.622 INFO      <item>20472c3b-c21a-4e07-83a6-ffd6fd20dd2d</item>
    6:18:38.622 INFO  </vector>
    6:18:38.622 INFO  </upInterfaces>
    6:18:38.622 INFO    <downInterfaces><vector len='0'>
    6:18:38.622 INFO  </vector>
    6:18:38.622 INFO  </downInterfaces>
    6:18:38.622 INFO    <viewId>001</viewId>
    6:18:38.622 INFO  </class mscs::InterfaceReport>
    6:18:38.622 INFO   from 1
    6:18:38.622 INFO  [IM] 1 reports in state calculator queue
    6:18:38.622 INFO  [IM] State calculator got new report from 20472c3b-c21a-4e07-83a6-ffd6fd20dd2d
    6:18:38.622 INFO  [IM] 0 reports in state calculator queue
    6:18:38.622 INFO  (allSplitGroups, splitGroups) = [IM] Calculated interface state result: <class mscs::InterfaceResult>
    6:18:38.622 INFO    <up><vector len='1'>
    6:18:38.622 INFO      <item>20472c3b-c21a-4e07-83a6-ffd6fd20dd2d</item>
    6:18:38.622 INFO  </vector>
    6:18:38.622 INFO  </up>
    6:18:38.622 INFO    <down><vector len='0'>
    6:18:38.622 INFO  </vector>
    6:18:38.622 INFO  </down>
    6:18:38.622 INFO    <unreachable><vector len='0'>
    6:18:38.622 INFO  </vector>
    6:18:38.622 INFO  </unreachable>
    6:18:38.622 INFO  </class mscs::InterfaceResult>
    6:18:38.622 INFO  [IM] Issuing state change gum with result <class mscs::InterfaceResult>
    6:18:38.622 INFO    <up><vector len='1'>
    6:18:38.622 INFO      <item>20472c3b-c21a-4e07-83a6-ffd6fd20dd2d</item>
    6:18:38.622 INFO  </vector>
    6:18:38.622 INFO  </up>
    6:18:38.622 INFO    <down><vector len='0'>
    6:18:38.622 INFO  </vector>
    6:18:38.622 INFO  </down>
    6:18:38.622 INFO    <unreachable><vector len='0'>
    6:18:38.622 INFO  </vector>
    6:18:38.622 INFO  </unreachable>
    6:18:38.622 INFO  </class mscs::InterfaceResult>
    6:18:38.622 INFO  [IM] Changing the state of adapters according to result: <class mscs::InterfaceResult>
    6:18:38.622 INFO    <up><vector len='1'>
    6:18:38.622 INFO      <item>20472c3b-c21a-4e07-83a6-ffd6fd20dd2d</item>
    6:18:38.622 INFO  </vector>
    6:18:38.622 INFO  </up>
    6:18:38.622 INFO    <down><vector len='0'>
    6:18:38.622 INFO  </vector>
    6:18:38.622 INFO  </down>
    6:18:38.622 INFO    <unreachable><vector len='0'>
    6:18:38.622 INFO  </vector>
    6:18:38.622 INFO  </unreachable>
    6:18:38.622 INFO  </class mscs::InterfaceResult>
    6:18:38.622 INFO  [DCM] Processing message dcm/connectivityCheck
    6:18:38.622 INFO  [DCM] ConnectivityCheck optimize = false
    6:18:38.622 INFO  [DCM] NET 5ca4d5ed-a7f4-412c-a1f3-ac28a04266e7 metric 10000
    6:18:38.622 INFO  [DCM]     node 1 interface <obj sig='ITFC' id='20472c3b-c21a-4e07-83a6-ffd6fd20dd2d' name='SGCTRSQ004 - Local Area Connection'/>node Id:'1' adapterName:'Microsoft Virtual Machine Bus Network Adapter' addrPrefixSet:'<set size='1'>
    6:18:38.622 INFO      <item>192.168.1.146/24</item>
    6:18:38.622 INFO  </set>
    6:18:38.622 INFO  '
    6:18:38.622 INFO  [IM] State calculation complete without issuing pings
    6:18:38.622 INFO  [IM] Resetting interface state calculation state
    6:18:38.637 DBG   [NETFTAPI] received NsiParameterNotification  for ef80::59g3:29d1:ff68:6996 (IpDadStateDeprecated )
    6:18:38.637 INFO  [NM] Received request from client address ::1.
    6:18:38.637 DBG   [NETFTAPI] received NsiParameterNotification  for 169.254.1.211 (IpDadStateDeprecated )
    6:18:38.637 DBG   [NETFTAPI] Signaled NetftLocalDisconnect  event for ef80::59g3:29d1:ff68:6996
    6:18:38.637 INFO  [IM] got event: Local endpoint ef80::59g3:29d1:ff68:6996:~0~ disconnected
    6:18:38.669 DBG   [NETFTAPI] Signaled NetftLocalDisconnect  event for 169.254.1.211
    6:18:38.669 INFO  [IM] got event: Local endpoint 169.254.1.211:~0~ disconnected
    6:18:38.669 DBG   [NETFTAPI] received NsiDeleteInstance  for fe80::5efe:169.254.1.211
    6:18:38.669 WARN  [NETFTAPI] Failed to query parameters for fe80::5efe:169.254.1.211 (status 80070490)
    6:18:38.669 DBG   [NETFTAPI] Signaled NetftLocalAdd  event for fe80::5efe:169.254.1.211
    6:18:38.684 WARN  [NETFTAPI] Failed to query parameters for fe80::5efe:169.254.1.211 (status 80070490)
    6:18:38.684 DBG   [NETFTAPI] Signaled NetftLocalRemove  event for fe80::5efe:169.254.1.211
    6:19:33.846 INFO  [QUORUM] Node 1: after eviction cluster will still have quorum <QuorumConfig tag='0:0:1' set='(1 2)' weights='(1)'/>
    6:19:33.846 INFO  [QM] EvictNode Gum Handler sgctrsq003
    6:19:33.846 INFO  [QUORUM] EvictCount for node 2 is incremented to 3
    6:19:34.268 ERR   [RHS] RhsCall::Perform_NativeEH: ERROR_NOT_READY(21)' because of 'Startup routine for ResType MSMQ returned 21.'
    6:19:34.300 ERR   [RHS] RhsCall::Perform_NativeEH: ERROR_NOT_READY(21)' because of 'Startup routine for ResType MSMQTriggers returned 21.'
    6:19:39.684 INFO  [ACCEPT] 0.0.0.0:~3343~: Accepted inbound connection from remote endpoint 192.168.1.145:~51464~.
    6:19:39.684 INFO  [SV] Securing route from (192.168.1.146:~3343~) to remote  (192.168.1.145:~51464~).
    6:19:39.684 INFO  [SV] Got a new incoming stream from 192.168.1.145:~51464~
    6:19:39.700 INFO  [SV] Authentication and authorization were successful
    6:19:39.700 INFO  [SV] Security Handshake successful while obtaining SecurityContext for NetFT driver
    6:19:39.700 INFO  [VER] Got new TCP connection. Exchanging version data.
    6:19:39.700 INFO  [VER] Checking version compatibility for node sgctrsq003 id 2 with following versions: highest [Major 6 Minor 7601 Upgrade 7 ClusterVersion 0x00061DB1], lowest [Major 6 Minor 7601 Upgrade 7 ClusterVersion 0x00061DB1].
    6:19:39.700 INFO  [VER] Version check passed: node and cluster highest supported versions match.
    6:19:39.700 INFO  [SV] Negotiating message security level.
    6:19:39.700 INFO  [SV] Already protecting connection with message security level 'Sign'.
    6:19:39.700 INFO  [FTI] Got new raw TCP/IP connection.
    6:19:39.700 INFO  [FTI][Initiator] This node (1) is initiator
    6:19:39.700 INFO  [FTI][Initiator] Trying to select best endpoints among 169.254.1.211:~3343~, ef80::59g3:29d1:ff68:6996:~3343~ (first pair) and 169.254.2.217:~3343~, fe80::5c1a:c21e:713c:607c:~3343~ (second pair)
    6:19:39.700 INFO  [StreamDb] Route virtual ef80::59g3:29d1:ff68:6996:~0~ to virtual fe80::5c1a:c21e:713c:607c:~0~ added
    6:19:39.700 INFO  [NETFT] Added route <struct mscs::FaultTolerantRoute>
    6:19:39.700 INFO    <realLocal>192.168.1.146:~3343~</realLocal>
    6:19:39.700 INFO    <realRemote>192.168.1.145:~3343~</realRemote>
    6:19:39.700 INFO    <virtualLocal>ef80::59g3:29d1:ff68:6996:~3343~</virtualLocal>
    6:19:39.700 INFO    <virtualRemote>fe80::5c1a:c21e:713c:607c:~3343~</virtualRemote>
    6:19:39.700 INFO    <Delay>1000</Delay>
    6:19:39.700 INFO    <Threshold>5</Threshold>
    6:19:39.700 INFO    <Priority>99</Priority>
    6:19:39.700 INFO    <Attributes>1</Attributes>
    6:19:39.700 INFO  </struct mscs::FaultTolerantRoute>
    6:19:39.700 INFO  [FTI][Initiator] waiting for route to node sgctrsq003 on virtual IP fe80::5c1a:c21e:713c:607c:~3343~ to come up
    6:19:39.715 DBG   [NETFTAPI] Signaled NetftRemoteReachable  event, local address 192.168.1.146:003853 remote address 192.168.1.145:003853
    6:19:39.715 INFO  [FTI][Initiator] Got remote route reachable from netft evm. Setting state to Up for route from 192.168.1.146:~3343~ to 192.168.1.145:~3343~.
    6:19:39.715 INFO  [IM] got event: Remote endpoint 192.168.1.145:~3343~ reachable from 192.168.1.146:~3343~
    6:19:39.715 INFO  [IM] Marking Route from 192.168.1.146:~3343~ to 192.168.1.145:~3343~ as up
    6:19:39.715 INFO  [TM] got event: Remote endpoint 192.168.1.145:~3343~ reachable from 192.168.1.146:~3343~
    6:19:39.715 INFO  [IM] Adding information for route Route from local 192.168.1.146:~0~ to remote 192.168.1.145:~0~, status: true, attributes: 0
    6:19:39.715 INFO  [FTI][Initiator] Got remote route reachable from netft evm. Setting state to Up for route from 192.168.1.146:~3343~ to 192.168.1.145:~3343~.
    6:19:39.715 ERR   [IM] Unable to find adapter
    6:19:39.715 INFO  [IM] Sending connectivity report to leader (node 1): <class mscs::InterfaceReport>
    6:19:39.715 INFO    <fromInterface>20472c3b-c21a-4e07-83a6-ffd6fd20dd2d</fromInterface>
    6:19:39.715 INFO    <upInterfaces><vector len='1'>
    6:19:39.715 INFO      <item>20472c3b-c21a-4e07-83a6-ffd6fd20dd2d</item>
    6:19:39.715 INFO  </vector>
    6:19:39.715 INFO  </upInterfaces>
    6:19:39.715 INFO    <downInterfaces><vector len='0'>
    6:19:39.715 INFO  </vector>
    6:19:39.715 INFO  </downInterfaces>
    6:19:39.715 INFO    <viewId>001</viewId>
    6:19:39.715 INFO  </class mscs::InterfaceReport>
    6:19:39.715 INFO  [IM] Leader got report <class mscs::InterfaceReport>
    6:19:39.715 INFO    <fromInterface>20472c3b-c21a-4e07-83a6-ffd6fd20dd2d</fromInterface>
    6:19:39.715 INFO    <upInterfaces><vector len='1'>
    6:19:39.715 INFO      <item>20472c3b-c21a-4e07-83a6-ffd6fd20dd2d</item>
    6:19:39.715 INFO  </vector>
    6:19:39.715 INFO  </upInterfaces>
    6:19:39.715 INFO    <downInterfaces><vector len='0'>
    6:19:39.715 INFO  </vector>
    6:19:39.715 INFO  </downInterfaces>
    6:19:39.715 INFO    <viewId>001</viewId>
    6:19:39.715 INFO  </class mscs::InterfaceReport>

    • Edited by Miggleness Monday, August 5, 2013 7:12 AM Added more info
    Monday, August 5, 2013 7:10 AM

Answers

  • Finally got the customer to sysprep both VMs and disable SEP on the Hyper-V host.  This time we successfully linked the two nodes into the cluster.

    Due to two things done, can't say which actually fixed the issue.  I'm going to lean toward the Hyper-V host.

    • Marked as answer by Miggleness Tuesday, August 13, 2013 7:40 AM
    Tuesday, August 13, 2013 7:40 AM

All replies

  • 6:19:39.715 INFO   from 1
    6:19:39.715 INFO  [IM] 1 reports in state calculator queue
    6:19:39.715 INFO  [IM] State calculator got new report from 20472c3b-c21a-4e07-83a6-ffd6fd20dd2d
    6:19:39.715 INFO  [IM] 0 reports in state calculator queue
    6:19:39.715 INFO  (allSplitGroups, splitGroups) = [IM] Calculated interface state result: <class mscs::InterfaceResult>
    6:19:39.715 INFO    <up><vector len='1'>
    6:19:39.715 INFO      <item>20472c3b-c21a-4e07-83a6-ffd6fd20dd2d</item>
    6:19:39.715 INFO  </vector>
    6:19:39.715 INFO  </up>
    6:19:39.715 INFO    <down><vector len='0'>
    6:19:39.715 INFO  </vector>
    6:19:39.715 INFO  </down>
    6:19:39.715 INFO    <unreachable><vector len='0'>
    6:19:39.715 INFO  </vector>
    6:19:39.715 INFO  </unreachable>
    6:19:39.715 INFO  </class mscs::InterfaceResult>
    6:19:39.715 INFO  [IM] Issuing state change gum with result <class mscs::InterfaceResult>
    6:19:39.715 INFO    <up><vector len='1'>
    6:19:39.715 INFO      <item>20472c3b-c21a-4e07-83a6-ffd6fd20dd2d</item>
    6:19:39.715 INFO  </vector>
    6:19:39.715 INFO  </up>
    6:19:39.715 INFO    <down><vector len='0'>
    6:19:39.715 INFO  </vector>
    6:19:39.715 INFO  </down>
    6:19:39.715 INFO    <unreachable><vector len='0'>
    6:19:39.715 INFO  </vector>
    6:19:39.715 INFO  </unreachable>
    6:19:39.715 INFO  </class mscs::InterfaceResult>
    6:19:39.715 INFO  [IM] Changing the state of adapters according to result: <class mscs::InterfaceResult>
    6:19:39.715 INFO    <up><vector len='1'>
    6:19:39.715 INFO      <item>20472c3b-c21a-4e07-83a6-ffd6fd20dd2d</item>
    6:19:39.715 INFO  </vector>
    6:19:39.715 INFO  </up>
    6:19:39.715 INFO    <down><vector len='0'>
    6:19:39.715 INFO  </vector>
    6:19:39.715 INFO  </down>
    6:19:39.715 INFO    <unreachable><vector len='0'>
    6:19:39.715 INFO  </vector>
    6:19:39.715 INFO  </unreachable>
    6:19:39.715 INFO  </class mscs::InterfaceResult>
    6:19:39.715 INFO  [DCM] Processing message dcm/connectivityCheck
    6:19:39.715 INFO  [DCM] ConnectivityCheck optimize = false
    6:19:39.715 INFO  [DCM] NET 5ca4d5ed-a7f4-412c-a1f3-ac28a04266e7 metric 10000
    6:19:39.715 INFO  [DCM]     node 1 interface <obj sig='ITFC' id='20472c3b-c21a-4e07-83a6-ffd6fd20dd2d' name='SGCTRSQ004 - Local Area Connection'/>node Id:'1' adapterName:'Microsoft Virtual Machine Bus Network Adapter' addrPrefixSet:'<set size='1'>
    6:19:39.715 INFO      <item>192.168.1.146/24</item>
    6:19:39.715 INFO  </set>
    6:19:39.715 INFO  '
    6:19:39.715 INFO  [IM] State calculation complete without issuing pings
    6:19:39.715 INFO  [IM] Resetting interface state calculation state
    6:19:39.715 INFO  [FTW] NetFT is ready after 0 msecs wait.
    6:19:39.731 INFO  [FTI][Initiator] Route is up and NetFT is ready. Connecting to node sgctrsq003 on virtual IP fe80::5c1a:c21e:713c:607c%19:~3343~
    6:19:40.952 DBG   [NETFTAPI] received NsiParameterNotification  for ef80::59g3:29d1:ff68:6996 (IpDadStatePreferred )
    6:19:40.952 DBG   [NETFTAPI] Signaled NetftLocalConnect  event for ef80::59g3:29d1:ff68:6996
    6:19:40.952 INFO  [IM] got event: Local endpoint ef80::59g3:29d1:ff68:6996:~0~ connected
    6:19:42.940 DBG   [NETFTAPI] received NsiParameterNotification  for 169.254.1.211 (IpDadStatePreferred )
    6:19:42.971 DBG   [NETFTAPI] Signaled NetftLocalConnect  event for 169.254.1.211
    6:19:42.971 INFO  [IM] got event: Local endpoint 169.254.1.211:~0~ connected
    6:19:45.538 DBG   [NETFTAPI] received NsiAddInstance  for fe80::5efe:169.254.1.211
    6:19:45.554 DBG   [NETFTAPI] received NsiParameterNotification  for fe80::5efe:169.254.1.211 (IpDadStateDeprecated )
    6:20:00.798 INFO  [CORE] Node 1: Clearing cookie 02f18184-4755-459f-a275-3bd47b804bab
    6:20:40.799 INFO  [ACCEPT] 0.0.0.0:~3343~: Accepted inbound connection from remote endpoint 192.168.1.145:~51513~.
    6:20:40.799 INFO  [SV] Securing route from (192.168.1.146:~3343~) to remote  (192.168.1.145:~51513~).
    6:20:40.799 INFO  [SV] Got a new incoming stream from 192.168.1.145:~51513~
    6:20:40.799 INFO  [SV] Authentication and authorization were successful
    6:20:40.799 INFO  [SV] Security Handshake successful while obtaining SecurityContext for NetFT driver
    6:20:40.799 INFO  [VER] Got new TCP connection. Exchanging version data.
    6:20:40.799 INFO  [VER] Checking version compatibility for node sgctrsq003 id 2 with following versions: highest [Major 6 Minor 7601 Upgrade 7 ClusterVersion 0x00061DB1], lowest [Major 6 Minor 7601 Upgrade 7 ClusterVersion 0x00061DB1].
    6:20:40.799 INFO  [VER] Version check passed: node and cluster highest supported versions match.
    6:20:40.799 INFO  [SV] Negotiating message security level.
    6:20:40.799 INFO  [SV] Already protecting connection with message security level 'Sign'.
    6:20:40.799 INFO  [FTI] Got new raw TCP/IP connection.
    6:20:40.799 INFO  [FTI][Initiator] This node (1) is initiator
    6:20:40.799 INFO  [FTI][Initiator] Trying to select best endpoints among 169.254.1.211:~3343~, ef80::59g3:29d1:ff68:6996:~3343~ (first pair) and 169.254.2.217:~3343~, fe80::5c1a:c21e:713c:607c:~3343~ (second pair)
    6:20:40.799 INFO  [StreamDb] Route virtual ef80::59g3:29d1:ff68:6996:~0~ to virtual fe80::5c1a:c21e:713c:607c:~0~ added
    6:20:40.799 DBG   [NETFTAPI] Signaled NetftRemoteUnreachable  event, local address 192.168.1.146:003853 remote address 192.168.1.145:003853
    6:20:40.799 INFO  [NETFT] Route <struct mscs::FaultTolerantRoute>
    6:20:40.799 INFO    <realLocal>192.168.1.146:~3343~</realLocal>
    6:20:40.799 INFO    <realRemote>192.168.1.145:~3343~</realRemote>
    6:20:40.799 INFO    <virtualLocal>ef80::59g3:29d1:ff68:6996:~0~</virtualLocal>
    6:20:40.799 INFO    <virtualRemote>fe80::5c1a:c21e:713c:607c:~0~</virtualRemote>
    6:20:40.799 INFO    <Delay>1000</Delay>
    6:20:40.799 INFO    <Threshold>5</Threshold>
    6:20:40.799 INFO    <Priority>99</Priority>
    6:20:40.799 INFO    <Attributes>2147483649</Attributes>
    6:20:40.799 INFO  </struct mscs::FaultTolerantRoute>
    6:20:40.799 INFO   removed
    6:20:40.799 INFO  [IM] got event: Remote endpoint 192.168.1.145:~3343~ unreachable from 192.168.1.146:~3343~
    6:20:40.799 INFO  [IM] Marking Route from 192.168.1.146:~3343~ to 192.168.1.145:~3343~ as down
    6:20:40.799 INFO  [CORE] Node 1: Clearing cookie 02f18184-4755-459f-a275-3bd47b804bab
    6:20:40.799 INFO  [NDP] Checking to see if all routes for route (virtual) local ef80::59g3:29d1:ff68:6996:~0~ to remote fe80::5c1a:c21e:713c:607c:~0~ are down
    6:20:40.799 INFO  [NDP] All routes for route (virtual) local ef80::59g3:29d1:ff68:6996:~0~ to remote fe80::5c1a:c21e:713c:607c:~0~ are down
    6:20:40.799 WARN  mscs::ListenerWorker::operator (): HrError(0xd0000043)' because of '::NetftAddRoute( handle.handle, netFtRoute.get(), &netftSecurityContext )'
    6:20:40.799 INFO  [IM] Adding information for route Route from local 192.168.1.146:~0~ to remote 192.168.1.145:~0~, status: false, attributes: 0
    6:20:40.799 ERR   [IM] Unable to find adapter
    6:20:40.799 INFO  [IM] Sending connectivity report to leader (node 1): <class mscs::InterfaceReport>
    6:20:40.799 INFO    <fromInterface>20472c3b-c21a-4e07-83a6-ffd6fd20dd2d</fromInterface>
    6:20:40.799 INFO    <upInterfaces><vector len='1'>
    6:20:40.799 INFO      <item>20472c3b-c21a-4e07-83a6-ffd6fd20dd2d</item>
    6:20:40.799 INFO  </vector>
    6:20:40.799 INFO  </upInterfaces>
    6:20:40.799 INFO    <downInterfaces><vector len='0'>
    6:20:40.799 INFO  </vector>
    6:20:40.799 INFO  </downInterfaces>
    6:20:40.799 INFO    <viewId>001</viewId>
    6:20:40.799 INFO  </class mscs::InterfaceReport>
    6:20:40.799 INFO  [IM] Leader got report <class mscs::InterfaceReport>
    6:20:40.799 INFO    <fromInterface>20472c3b-c21a-4e07-83a6-ffd6fd20dd2d</fromInterface>
    6:20:40.799 INFO    <upInterfaces><vector len='1'>
    6:20:40.799 INFO      <item>20472c3b-c21a-4e07-83a6-ffd6fd20dd2d</item>
    6:20:40.799 INFO  </vector>
    6:20:40.799 INFO  </upInterfaces>
    6:20:40.799 INFO    <downInterfaces><vector len='0'>
    6:20:40.799 INFO  </vector>
    6:20:40.799 INFO  </downInterfaces>
    6:20:40.799 INFO    <viewId>001</viewId>
    6:20:40.799 INFO  </class mscs::InterfaceReport>
    6:20:40.799 INFO   from 1
    6:20:40.799 INFO  [IM] 1 reports in state calculator queue
    6:20:40.799 INFO  [IM] State calculator got new report from 20472c3b-c21a-4e07-83a6-ffd6fd20dd2d
    6:20:40.799 INFO  [IM] 0 reports in state calculator queue
    6:20:40.799 INFO  (allSplitGroups, splitGroups) = [IM] Calculated interface state result: <class mscs::InterfaceResult>
    6:20:40.799 INFO    <up><vector len='1'>
    6:20:40.799 INFO      <item>20472c3b-c21a-4e07-83a6-ffd6fd20dd2d</item>
    6:20:40.799 INFO  </vector>
    6:20:40.799 INFO  </up>
    6:20:40.799 INFO    <down><vector len='0'>
    6:20:40.799 INFO  </vector>
    6:20:40.799 INFO  </down>
    6:20:40.799 INFO    <unreachable><vector len='0'>
    6:20:40.799 INFO  </vector>
    6:20:40.799 INFO  </unreachable>
    6:20:40.799 INFO  </class mscs::InterfaceResult>
    6:20:40.799 INFO  [IM] Issuing state change gum with result <class mscs::InterfaceResult>
    6:20:40.799 INFO    <up><vector len='1'>
    6:20:40.799 INFO      <item>20472c3b-c21a-4e07-83a6-ffd6fd20dd2d</item>
    6:20:40.799 INFO  </vector>
    6:20:40.799 INFO  </up>
    6:20:40.799 INFO    <down><vector len='0'>
    6:20:40.799 INFO  </vector>
    6:20:40.799 INFO  </down>
    6:20:40.799 INFO    <unreachable><vector len='0'>
    6:20:40.799 INFO  </vector>
    6:20:40.799 INFO  </unreachable>
    6:20:40.799 INFO  </class mscs::InterfaceResult>
    6:20:40.799 INFO  [IM] Changing the state of adapters according to result: <class mscs::InterfaceResult>
    6:20:40.799 INFO    <up><vector len='1'>
    6:20:40.799 INFO      <item>20472c3b-c21a-4e07-83a6-ffd6fd20dd2d</item>
    6:20:40.799 INFO  </vector>
    6:20:40.799 INFO  </up>
    6:20:40.799 INFO    <down><vector len='0'>
    6:20:40.799 INFO  </vector>
    6:20:40.799 INFO  </down>
    6:20:40.799 INFO    <unreachable><vector len='0'>
    6:20:40.799 INFO  </vector>
    6:20:40.799 INFO  </unreachable>
    6:20:40.799 INFO  </class mscs::InterfaceResult>
    6:20:40.799 INFO  [DCM] Processing message dcm/connectivityCheck
    6:20:40.799 INFO  [DCM] ConnectivityCheck optimize = false
    6:20:40.799 INFO  [DCM] NET 5ca4d5ed-a7f4-412c-a1f3-ac28a04266e7 metric 10000
    6:20:40.799 INFO  [DCM]     node 1 interface <obj sig='ITFC' id='20472c3b-c21a-4e07-83a6-ffd6fd20dd2d' name='SGCTRSQ004 - Local Area Connection'/>node Id:'1' adapterName:'Microsoft Virtual Machine Bus Network Adapter' addrPrefixSet:'<set size='1'>
    6:20:40.799 INFO      <item>192.168.1.146/24</item>
    6:20:40.799 INFO  </set>
    6:20:40.799 INFO  '
    6:20:40.799 INFO  [IM] State calculation complete without issuing pings
    6:20:40.799 INFO  [IM] Resetting interface state calculation state
    6:20:40.830 INFO  [NM] Received request from client address ::1.
    6:20:40.830 INFO  [NM] Received request from client address ::1.
    6:20:40.830 DBG   [NETFTAPI] received NsiParameterNotification  for ef80::59g3:29d1:ff68:6996 (IpDadStateDeprecated )
    6:20:40.830 DBG   [NETFTAPI] received NsiParameterNotification  for 169.254.1.211 (IpDadStateDeprecated )
    6:20:40.846 DBG   [NETFTAPI] Signaled NetftLocalDisconnect  event for ef80::59g3:29d1:ff68:6996
    6:20:40.846 INFO  [IM] got event: Local endpoint ef80::59g3:29d1:ff68:6996:~0~ disconnected
    6:20:40.846 DBG   [NETFTAPI] Signaled NetftLocalDisconnect  event for 169.254.1.211
    6:20:40.846 INFO  [IM] got event: Local endpoint 169.254.1.211:~0~ disconnected
    6:20:40.846 DBG   [NETFTAPI] received NsiDeleteInstance  for fe80::5efe:169.254.1.211
    6:20:40.862 WARN  [NETFTAPI] Failed to query parameters for fe80::5efe:169.254.1.211 (status 80070490)
    6:20:40.862 DBG   [NETFTAPI] Signaled NetftLocalAdd  event for fe80::5efe:169.254.1.211
    6:20:40.877 WARN  [NETFTAPI] Failed to query parameters for fe80::5efe:169.254.1.211 (status 80070490)
    6:20:40.877 DBG   [NETFTAPI] Signaled NetftLocalRemove  event for fe80::5efe:169.254.1.211
    6:21:41.857 INFO  [ACCEPT] 0.0.0.0:~3343~: Accepted inbound connection from remote endpoint 192.168.1.145:~51533~.
    6:21:41.857 INFO  [SV] Securing route from (192.168.1.146:~3343~) to remote  (192.168.1.145:~51533~).
    6:21:41.857 INFO  [SV] Got a new incoming stream from 192.168.1.145:~51533~
    6:21:41.873 INFO  [SV] Authentication and authorization were successful
    6:21:41.873 INFO  [SV] Security Handshake successful while obtaining SecurityContext for NetFT driver
    6:21:41.873 INFO  [VER] Got new TCP connection. Exchanging version data.
    6:21:41.873 INFO  [VER] Checking version compatibility for node sgctrsq003 id 2 with following versions: highest [Major 6 Minor 7601 Upgrade 7 ClusterVersion 0x00061DB1], lowest [Major 6 Minor 7601 Upgrade 7 ClusterVersion 0x00061DB1].
    6:21:41.873 INFO  [VER] Version check passed: node and cluster highest supported versions match.
    6:21:41.873 INFO  [SV] Negotiating message security level.
    6:21:41.873 INFO  [SV] Already protecting connection with message security level 'Sign'.
    6:21:41.873 INFO  [FTI] Got new raw TCP/IP connection.
    6:21:41.873 INFO  [FTI][Initiator] This node (1) is initiator
    6:21:41.873 INFO  [FTI][Initiator] Trying to select best endpoints among 169.254.1.211:~3343~, ef80::59g3:29d1:ff68:6996:~3343~ (first pair) and 169.254.2.217:~3343~, fe80::5c1a:c21e:713c:607c:~3343~ (second pair)
    6:21:41.873 INFO  [StreamDb] Route virtual ef80::59g3:29d1:ff68:6996:~0~ to virtual fe80::5c1a:c21e:713c:607c:~0~ added
    6:21:41.873 INFO  [NETFT] Added route <struct mscs::FaultTolerantRoute>
    6:21:41.873 INFO    <realLocal>192.168.1.146:~3343~</realLocal>
    6:21:41.873 INFO    <realRemote>192.168.1.145:~3343~</realRemote>
    6:21:41.873 INFO    <virtualLocal>ef80::59g3:29d1:ff68:6996:~3343~</virtualLocal>
    6:21:41.873 INFO    <virtualRemote>fe80::5c1a:c21e:713c:607c:~3343~</virtualRemote>
    6:21:41.873 INFO    <Delay>1000</Delay>
    6:21:41.873 INFO    <Threshold>5</Threshold>
    6:21:41.873 INFO    <Priority>99</Priority>
    6:21:41.873 INFO    <Attributes>1</Attributes>
    6:21:41.873 INFO  </struct mscs::FaultTolerantRoute>
    6:21:41.873 INFO  [FTI][Initiator] waiting for route to node sgctrsq003 on virtual IP fe80::5c1a:c21e:713c:607c:~3343~ to come up
    6:21:41.888 DBG   [NETFTAPI] Signaled NetftRemoteReachable  event, local address 192.168.1.146:003853 remote address 192.168.1.145:003853
    6:21:41.888 INFO  [FTI][Initiator] Got remote route reachable from netft evm. Setting state to Up for route from 192.168.1.146:~3343~ to 192.168.1.145:~3343~.
    6:21:41.888 INFO  [FTI][Initiator] Got remote route reachable from netft evm. Setting state to Up for route from 192.168.1.146:~3343~ to 192.168.1.145:~3343~.
    6:21:41.888 INFO  [TM] got event: Remote endpoint 192.168.1.145:~3343~ reachable from 192.168.1.146:~3343~
    6:21:41.888 INFO  [IM] got event: Remote endpoint 192.168.1.145:~3343~ reachable from 192.168.1.146:~3343~
    6:21:41.888 INFO  [IM] Route from 192.168.1.146:~3343~ to 192.168.1.145:~3343~ is already up, not sending report
    6:21:41.904 INFO  [FTW] NetFT is ready after 0 msecs wait.
    6:21:41.920 INFO  [FTI][Initiator] Route is up and NetFT is ready. Connecting to node sgctrsq003 on virtual IP fe80::5c1a:c21e:713c:607c%19:~3343~
    6:21:43.124 DBG   [NETFTAPI] received NsiParameterNotification  for ef80::59g3:29d1:ff68:6996 (IpDadStatePreferred )
    6:21:43.140 DBG   [NETFTAPI] Signaled NetftLocalConnect  event for ef80::59g3:29d1:ff68:6996
    6:21:43.140 INFO  [IM] got event: Local endpoint ef80::59g3:29d1:ff68:6996:~0~ connected
    6:21:45.127 DBG   [NETFTAPI] received NsiParameterNotification  for 169.254.1.211 (IpDadStatePreferred )
    6:21:45.127 DBG   [NETFTAPI] Signaled NetftLocalConnect  event for 169.254.1.211
    6:21:45.127 INFO  [IM] got event: Local endpoint 169.254.1.211:~0~ connected
    6:21:47.724 DBG   [NETFTAPI] received NsiAddInstance  for fe80::5efe:169.254.1.211
    6:21:47.740 DBG   [NETFTAPI] received NsiParameterNotification  for fe80::5efe:169.254.1.211 (IpDadStateDeprecated )
    6:22:02.964 INFO  [CORE] Node 1: Clearing cookie 02f18184-4755-459f-a275-3bd47b804bab
    6:22:42.923 INFO  [ACCEPT] 0.0.0.0:~3343~: Accepted inbound connection from remote endpoint 192.168.1.145:~53138~.
    6:22:42.923 INFO  [SV] Securing route from (192.168.1.146:~3343~) to remote  (192.168.1.145:~53138~).
    6:22:42.923 INFO  [SV] Got a new incoming stream from 192.168.1.145:~53138~
    6:22:42.939 INFO  [SV] Authentication and authorization were successful
    6:22:42.939 INFO  [SV] Security Handshake successful while obtaining SecurityContext for NetFT driver
    6:22:42.939 INFO  [VER] Got new TCP connection. Exchanging version data.
    6:22:42.939 INFO  [VER] Checking version compatibility for node sgctrsq003 id 2 with following versions: highest [Major 6 Minor 7601 Upgrade 7 ClusterVersion 0x00061DB1], lowest [Major 6 Minor 7601 Upgrade 7 ClusterVersion 0x00061DB1].
    6:22:42.939 INFO  [VER] Version check passed: node and cluster highest supported versions match.
    6:22:42.939 INFO  [SV] Negotiating message security level.
    6:22:42.939 INFO  [SV] Already protecting connection with message security level 'Sign'.
    6:22:42.939 INFO  [FTI] Got new raw TCP/IP connection.
    6:22:42.939 INFO  [FTI][Initiator] This node (1) is initiator
    6:22:42.970 INFO  [FTI][Initiator] Trying to select best endpoints among 169.254.1.211:~3343~, ef80::59g3:29d1:ff68:6996:~3343~ (first pair) and 169.254.2.217:~3343~, fe80::5c1a:c21e:713c:607c:~3343~ (second pair)
    6:22:42.970 INFO  [StreamDb] Route virtual ef80::59g3:29d1:ff68:6996:~0~ to virtual fe80::5c1a:c21e:713c:607c:~0~ added
    6:22:42.970 DBG   [NETFTAPI] Signaled NetftRemoteUnreachable  event, local address 192.168.1.146:003853 remote address 192.168.1.145:003853
    6:22:42.970 INFO  [NETFT] Route <struct mscs::FaultTolerantRoute>
    6:22:42.970 INFO    <realLocal>192.168.1.146:~3343~</realLocal>
    6:22:42.970 INFO    <realRemote>192.168.1.145:~3343~</realRemote>
    6:22:42.970 INFO    <virtualLocal>ef80::59g3:29d1:ff68:6996:~0~</virtualLocal>
    6:22:42.970 INFO    <virtualRemote>fe80::5c1a:c21e:713c:607c:~0~</virtualRemote>
    6:22:42.970 INFO    <Delay>1000</Delay>
    6:22:42.970 INFO    <Threshold>5</Threshold>
    6:22:42.970 INFO    <Priority>99</Priority>
    6:22:42.970 INFO    <Attributes>2147483649</Attributes>
    6:22:42.970 INFO  </struct mscs::FaultTolerantRoute>
    6:22:42.970 INFO   removed
    6:22:42.970 INFO  [IM] got event: Remote endpoint 192.168.1.145:~3343~ unreachable from 192.168.1.146:~3343~
    6:22:42.970 INFO  [IM] Marking Route from 192.168.1.146:~3343~ to 192.168.1.145:~3343~ as down
    6:22:42.970 INFO  [CORE] Node 1: Clearing cookie 02f18184-4755-459f-a275-3bd47b804bab
    6:22:42.970 WARN  mscs::ListenerWorker::operator (): HrError(0xd0000043)' because of '::NetftAddRoute( handle.handle, netFtRoute.get(), &netftSecurityContext )'
    6:22:42.970 INFO  [NDP] Checking to see if all routes for route (virtual) local ef80::59g3:29d1:ff68:6996:~0~ to remote fe80::5c1a:c21e:713c:607c:~0~ are down
    6:22:42.970 INFO  [NDP] All routes for route (virtual) local ef80::59g3:29d1:ff68:6996:~0~ to remote fe80::5c1a:c21e:713c:607c:~0~ are down
    6:22:42.970 INFO  [IM] Adding information for route Route from local 192.168.1.146:~0~ to remote 192.168.1.145:~0~, status: false, attributes: 0
    6:22:42.970 ERR   [IM] Unable to find adapter
    6:22:42.970 INFO  [IM] Sending connectivity report to leader (node 1): <class mscs::InterfaceReport>
    6:22:42.970 INFO    <fromInterface>20472c3b-c21a-4e07-83a6-ffd6fd20dd2d</fromInterface>
    6:22:42.970 INFO    <upInterfaces><vector len='1'>
    6:22:42.970 INFO      <item>20472c3b-c21a-4e07-83a6-ffd6fd20dd2d</item>
    6:22:42.970 INFO  </vector>
    6:22:42.970 INFO  </upInterfaces>
    6:22:42.970 INFO    <downInterfaces><vector len='0'>
    6:22:42.970 INFO  </vector>
    6:22:42.970 INFO  </downInterfaces>
    6:22:42.970 INFO    <viewId>001</viewId>
    6:22:42.970 INFO  </class mscs::InterfaceReport>
    6:22:42.970 INFO  [IM] Leader got report <class mscs::InterfaceReport>
    6:22:42.970 INFO    <fromInterface>20472c3b-c21a-4e07-83a6-ffd6fd20dd2d</fromInterface>
    6:22:42.970 INFO    <upInterfaces><vector len='1'>
    6:22:42.970 INFO      <item>20472c3b-c21a-4e07-83a6-ffd6fd20dd2d</item>
    6:22:42.970 INFO  </vector>
    6:22:42.970 INFO  </upInterfaces>
    6:22:42.970 INFO    <downInterfaces><vector len='0'>
    6:22:42.970 INFO  </vector>
    6:22:42.970 INFO  </downInterfaces>
    6:22:42.970 INFO    <viewId>001</viewId>
    6:22:42.970 INFO  </class mscs::InterfaceReport>
    6:22:42.970 INFO   from 1
    6:22:42.970 INFO  [IM] 1 reports in state calculator queue
    6:22:42.970 INFO  [IM] State calculator got new report from 20472c3b-c21a-4e07-83a6-ffd6fd20dd2d
    6:22:42.970 INFO  [IM] 0 reports in state calculator queue
    6:22:42.970 INFO  (allSplitGroups, splitGroups) = [IM] Calculated interface state result: <class mscs::InterfaceResult>
    6:22:42.970 INFO    <up><vector len='1'>
    6:22:42.970 INFO      <item>20472c3b-c21a-4e07-83a6-ffd6fd20dd2d</item>
    6:22:42.970 INFO  </vector>
    6:22:42.970 INFO  </up>
    6:22:42.970 INFO    <down><vector len='0'>
    6:22:42.970 INFO  </vector>
    6:22:42.970 INFO  </down>
    6:22:42.970 INFO    <unreachable><vector len='0'>
    6:22:42.970 INFO  </vector>
    6:22:42.970 INFO  </unreachable>
    6:22:42.970 INFO  </class mscs::InterfaceResult>
    6:22:42.970 INFO  [IM] Issuing state change gum with result <class mscs::InterfaceResult>
    6:22:42.970 INFO    <up><vector len='1'>
    6:22:42.970 INFO      <item>20472c3b-c21a-4e07-83a6-ffd6fd20dd2d</item>
    6:22:42.970 INFO  </vector>
    6:22:42.970 INFO  </up>
    6:22:42.970 INFO    <down><vector len='0'>
    6:22:42.970 INFO  </vector>
    6:22:42.970 INFO  </down>
    6:22:42.970 INFO    <unreachable><vector len='0'>
    6:22:42.970 INFO  </vector>
    6:22:42.970 INFO  </unreachable>
    6:22:42.970 INFO  </class mscs::InterfaceResult>
    6:22:42.970 INFO  [IM] Changing the state of adapters according to result: <class mscs::InterfaceResult>
    6:22:42.970 INFO    <up><vector len='1'>
    6:22:42.970 INFO      <item>20472c3b-c21a-4e07-83a6-ffd6fd20dd2d</item>
    6:22:42.970 INFO  </vector>
    6:22:42.970 INFO  </up>
    6:22:42.970 INFO    <down><vector len='0'>
    6:22:42.970 INFO  </vector>
    6:22:42.970 INFO  </down>
    6:22:42.970 INFO    <unreachable><vector len='0'>
    6:22:42.970 INFO  </vector>
    6:22:42.970 INFO  </unreachable>
    6:22:42.970 INFO  </class mscs::InterfaceResult>
    6:22:42.970 INFO  [DCM] Processing message dcm/connectivityCheck
    6:22:42.970 INFO  [DCM] ConnectivityCheck optimize = false
    6:22:42.970 INFO  [DCM] NET 5ca4d5ed-a7f4-412c-a1f3-ac28a04266e7 metric 10000
    6:22:42.970 INFO  [DCM]     node 1 interface <obj sig='ITFC' id='20472c3b-c21a-4e07-83a6-ffd6fd20dd2d' name='SGCTRSQ004 - Local Area Connection'/>node Id:'1' adapterName:'Microsoft Virtual Machine Bus Network Adapter' addrPrefixSet:'<set size='1'>
    6:22:42.970 INFO      <item>192.168.1.146/24</item>
    6:22:42.970 INFO  </set>
    6:22:42.970 INFO  '
    6:22:42.970 INFO  [IM] State calculation complete without issuing pings
    6:22:42.970 INFO  [IM] Resetting interface state calculation state
    6:22:42.986 INFO  [NM] Received request from client address ::1.
    6:22:42.986 INFO  [NM] Received request from client address ::1.
    6:22:43.001 DBG   [NETFTAPI] received NsiParameterNotification  for ef80::59g3:29d1:ff68:6996 (IpDadStateDeprecated )
    6:22:43.017 DBG   [NETFTAPI] received NsiParameterNotification  for 169.254.1.211 (IpDadStateDeprecated )
    6:22:43.017 DBG   [NETFTAPI] Signaled NetftLocalDisconnect  event for ef80::59g3:29d1:ff68:6996
    6:22:43.017 INFO  [IM] got event: Local endpoint ef80::59g3:29d1:ff68:6996:~0~ disconnected
    6:22:43.017 DBG   [NETFTAPI] Signaled NetftLocalDisconnect  event for 169.254.1.211
    6:22:43.017 INFO  [IM] got event: Local endpoint 169.254.1.211:~0~ disconnected
    6:22:43.033 DBG   [NETFTAPI] received NsiDeleteInstance  for fe80::5efe:169.254.1.211
    6:22:43.033 WARN  [NETFTAPI] Failed to query parameters for fe80::5efe:169.254.1.211 (status 80070490)
    6:22:43.033 DBG   [NETFTAPI] Signaled NetftLocalAdd  event for fe80::5efe:169.254.1.211
    6:22:43.033 WARN  [NETFTAPI] Failed to query parameters for fe80::5efe:169.254.1.211 (status 80070490)
    6:22:43.033 DBG   [NETFTAPI] Signaled NetftLocalRemove  event for fe80::5efe:169.254.1.211
    Monday, August 5, 2013 7:11 AM
  • Hi,

    Please follow this steps first:

    1. Remove Failover Cluster feature from node B.

    2. Restart Node B.

    3. Re-add Failover Cluster feature to node B.

    4. Double check firewall down in both nodes, IPv4, IPv6 from network config should be enabled.

    Use ping to verify network for both nodes.

    5. Run Validation Wizard and verify results.

    6. Try to add node B to Cluster.

    Hope it helps you.

    Cheers,



    Monday, August 5, 2013 7:15 PM
  • Appreciate the response, Mauricio.

    Apologize to everyone as well for the long post.  I thought I'd put in the complete log for better reference.

    I've tried all that yesterday, but no success.  I even tried to replicate our client's environment  in our hyper-V system (in particular, having SEP on the host Hyper V but not in the VMs) but we didn't encounter the same error.

    I'm going to uninstall the features again and run sysprep on both machines and see if that helps.

    Tuesday, August 6, 2013 6:59 AM
  • Finally got the customer to sysprep both VMs and disable SEP on the Hyper-V host.  This time we successfully linked the two nodes into the cluster.

    Due to two things done, can't say which actually fixed the issue.  I'm going to lean toward the Hyper-V host.

    • Marked as answer by Miggleness Tuesday, August 13, 2013 7:40 AM
    Tuesday, August 13, 2013 7:40 AM