none
How to add RDMs to VMware SQL Cluster

    Question

  • I’m trying to add RDMs for HP 3PAR SAN storage onto a SQL 2012 cluster of VMware VMs.  Although I did a lot of research and am following a nicely documented method (“How-To: Migrate MS SQL Cluster to a New SAN”), it just isn’t working.

    We have two VMs running as SQL cluster nodes.  They have vmdk files for their C: and E: drives, and RDM files (on our old EMC SAN) for the cluster drives (Quorum, DTC, SQL data & log volumes).  The cluster nodes and their SQL databases all work fine, as does failover.   All the files for one VM (vmdk and old RDM pointers) are located in that server’s folder on one datashare, and all the files for the other VM are in the second VM’s folder on another datashare; and the SCSI controllers are both set as non-shared.  This does look slightly strange (to my untutored eye) as it is NOT how the internet method instructs for adding new disks – and after completing the instructions I get errors when starting the VMs.

    The internet instructions say to create thin LUNs (except thick for the new Quorum drive).  Then (presumably with both VMs off) set the SCSI controllers to shared physical, add disks (as new RDM disks) to one VM, and note the disk-filenames generated.  Then add them to the other VM (as existing disks, browse to the same files).  Then restart both VMs and rescan disks on one, create volumes, create cluster disks, test failover etc.  The VMs won’t restart, though, vSphere shows the message “Thin/TBZ/Sparse disks cannot be opened in multiwriter mode”.  In order to get these production machines running again for our users, I remove the added disks, but the machines will only start if I reset the SCSI controllers as non-shared again.

    Can anyone help me escape from this quandary, please?  Should the SCSI controllers be non-shared (as working now), or shared physical (as per instructions)?  Should I be creating files separately for each VM’s new disks in that VM’s datastore folder (as working now) or re-using newly-created files (as in the How-to)?  The latest best-practices guide is great on principles, but less so on how-to.

    TIA

    Ian

    Wednesday, April 5, 2017 9:32 AM

Answers

  • Solution found! - After much trial & error.  This works for ESXi 5.1 hosts / Windows 2008 R2 guests / MS SQL 2012 / MSCS Clustering

    • In Failover Cluster Manager, offline SQL resources and shutdown both nodes. (This may only be required if creating new SCSI controllers & needing to set bus sharing - you can't do anything with controllers if the VM is running.)  Other internet guidance isn't clear about this.
    • In vSphere, choose first node, Edit Settings, add Hard Disk, choose RDM, select LUN, store with VM, Physical mode, choose a SCSI Id on a new controller (e.g. 2:0).  I chose to use a new set of Ids because I am migrating SQL files (also Quorum & DTC disks) from an old SAN to a new one, and will eventually delete old RDM disks residing on the old SAN.
    • This creates a new SCSI controller - set it to SCSI Bus Sharing as Physical.
    • Add more hard disks as necessary, choosing SCSI Ids on the new controller.
    • Note the filenames & locations of the RDM mapping file for each new disk.
    • Now choose the second cluster node in vSphere, Edit Settings, add Hard Disk, Use Existing Disk, browse to the first file/location noted earlier, choose matching SCSI Id (2:0)
    • Again a new SCSI controller is created - set its SCSI Bus sharing to Physical.
    • Add remaining hard disks, in each case matching up the filenames/locations/SCSI Ids with the values for the first node.
    • Restart both nodes.
    • On one node, go to Disk Management, Rescan Disks.  Online the disks, initialise them, and create new volumes.
    • In Failover Cluster Manager, online SQL resources again, create Cluster Disks and assign them as required.
    • Test failover & check the new disks migrate between nodes.
    • Marked as answer by Ian Button Wednesday, April 12, 2017 7:28 AM
    Wednesday, April 12, 2017 7:28 AM

All replies

  • Additional Info

    VMware is ESXi 5.1 U3.  Guest OS is Windows Server 2008 R2.   SQL version is SQL Server 2012 (two instances).

    Both old & new SANs are FC connected to the VMware hosts, so using RDMs in physical mode is correct  (VMware KB2147662 : “In vSphere 5.1 and earlier, configurations using shared storage for Quorum and/or Data must be on FC based RDMs (physical mode for Cluster Across Boxes, virtual mode for Cluster In A Box”).  We have anti-affinity rules to keep the two VMs on separate hosts (CAB).

    Wednesday, April 5, 2017 2:18 PM
  • This question is not directly related to SQL Server but to storage and Vmware your best bet is to ask in forum related to VMware

    Cheers,

    Shashank

    Please mark this reply as answer if it solved your issue or vote as helpful if it helped so that other forum members can benefit from it

    My TechNet Wiki Articles

    MVP

    Wednesday, April 5, 2017 4:16 PM
    Moderator
  • you can refer following link

    https://www.vmadmin.co.uk/resources/35-esxserver/58-rdmvm


    Please Mark it as Answered if it answered your question OR mark it as Helpful if it help you to solve your problem.

    Sunday, April 9, 2017 7:40 AM
  • Solution found! - After much trial & error.  This works for ESXi 5.1 hosts / Windows 2008 R2 guests / MS SQL 2012 / MSCS Clustering

    • In Failover Cluster Manager, offline SQL resources and shutdown both nodes. (This may only be required if creating new SCSI controllers & needing to set bus sharing - you can't do anything with controllers if the VM is running.)  Other internet guidance isn't clear about this.
    • In vSphere, choose first node, Edit Settings, add Hard Disk, choose RDM, select LUN, store with VM, Physical mode, choose a SCSI Id on a new controller (e.g. 2:0).  I chose to use a new set of Ids because I am migrating SQL files (also Quorum & DTC disks) from an old SAN to a new one, and will eventually delete old RDM disks residing on the old SAN.
    • This creates a new SCSI controller - set it to SCSI Bus Sharing as Physical.
    • Add more hard disks as necessary, choosing SCSI Ids on the new controller.
    • Note the filenames & locations of the RDM mapping file for each new disk.
    • Now choose the second cluster node in vSphere, Edit Settings, add Hard Disk, Use Existing Disk, browse to the first file/location noted earlier, choose matching SCSI Id (2:0)
    • Again a new SCSI controller is created - set its SCSI Bus sharing to Physical.
    • Add remaining hard disks, in each case matching up the filenames/locations/SCSI Ids with the values for the first node.
    • Restart both nodes.
    • On one node, go to Disk Management, Rescan Disks.  Online the disks, initialise them, and create new volumes.
    • In Failover Cluster Manager, online SQL resources again, create Cluster Disks and assign them as required.
    • Test failover & check the new disks migrate between nodes.
    • Marked as answer by Ian Button Wednesday, April 12, 2017 7:28 AM
    Wednesday, April 12, 2017 7:28 AM