locked
How to build a large CIFS server with high availability in Azure RRS feed

  • Question

  • Hi,

    I am trying to see if it would be viable to move hundreds of TB of data sitting on SMB shares on site to Azure.

    Requirements:

    • Storage capacity 400TB+
    • Accessed by multiple users from multiple offices so must be able to implement permissions on users and groups
    • Integrate with AD
    • RPO up to 24 hours
    • RTO up to 24 hours
    • Availability 99.95%

    On site it sits on traditional MS Windows VMs hosted on VMware and Hyper-V based clusters with storage from SANs.

    The availability of data is very high currently but I have been asked to reduce the IT Infrastructure footprint on site and have been trying to figure out how to serve this data from Azure and maintain a 99.95+ availability.

    I started thinking in traditional terms of building a Windows Server cluster in Azure and then attaching 100s TB worth of blob storage. But then ran into the problem that you can't get block storage(highly available) in Azure to allow you to build a Windows Server Cluster.

    A single VM with 100's TB of Blob storage has no SLA in Azure and can be taken down for various reasons including host maintenance or Windows Server security patching so this would not be an option.

    I trialed a StorSimple appliance onsite however this had its limitations such as incredibly slow to snapshot 50TB+ of data taking the better part of 24 hours and breaking the 24 hr RPO associated with the data. On top of that users are located in many different offices and then need to come into the DC where the Storsimple is located and potentially back out to Azure. It would be much more efficient for them to break out of the offices straight out to Azure.

    I considered using the Storsimple virtual appliance in Azure as block storage for the Windows Cluster however the appliance is also a single VM I believe and it would also not be covered by the Azure availability SLA.

    I considered Azure File Services but these don't support ACLs and can only be mapped to a VM in the same Azure subscription I believe. They are also limited to 5TB per share.

    Has anyone successfully found a cost effective way to build large capacity and highly available CIFS storage in Azure?

    Thanks

    Friday, June 3, 2016 8:38 AM

All replies

  • Hello,

    We are checking on the query and would get back to you soon on this.
    I apologize for the inconvenience and appreciate your time and patience in this matter.

    Regards,

    Sapna Girish


    Sapna G

    Friday, June 3, 2016 7:21 PM
  • Hi,

    Thank you for posting here!

    If you want to move on-premises data to Azure Storage (or vice versa), there are a variety of ways to do this. The approach that works best for you will depend on your scenario. Below mentioned link will provide a quick overview of different scenarios and appropriate offerings for each one.

    https://azure.microsoft.com/en-in/documentation/articles/storage-moving-data/

    In StorSimple, the maximum size of a Volume is 64TB and the Maximum number of volumes you can have are 255 which approximates to 16PB of data. Also in an Azure Storage account you can store up to 500 TB of data and every subscription can have up to 100 Storage Accounts, which approximates to 50 PB of data.

    . You could refer the following link for StorSimple System limits: https://azure.microsoft.com/en-us/documentation/articles/azure-subscription-service-limits/#storsimple-system-limits  

    .  For Azure Storage Scalability and Performance Targets refer below link:

    https://azure.microsoft.com/en-in/documentation/articles/storage-scalability-targets/

    .  To implement permissions on users and groups with Role-Based Access Control (RBAC) refer below   link:

    https://azure.microsoft.com/en-in/documentation/articles/storage-security-guide/#management-plane-security  

    . Guaranteed 99.9% or greater availability (excludes preview services). Read the SLA

    Also, to avoid downtime due to Planned or unplanned Maintenance, the best approach would be to create your VMs in an availability set.

    You could refer the following link for details to set up and manage multiple virtual machines to ensure high availability: https://azure.microsoft.com/en-in/documentation/articles/virtual-machines-windows-manage-availability/

    Regards,

    Vikranth S.

    Please remember to click "Mark as Answer" on the post that helps you, this can be beneficial to other community members reading the thread. And Vote as Helpful.

    Monday, June 6, 2016 8:22 AM
  • You can use a 3rd party software defined storage provider like StarWind Virtual SAN https://www.starwindsoftware.com/starwind-virtual-san. Just install the software on both VM’s that hold your HA data. The tool will mirror your data between both VMs provided this mirrored highly available and fault-tolerant disk as a block-level storage for your cluster needs. Here is a guide for SQL server https://www.starwindsoftware.com/installing-and-configuring-a-sql-server-fci-on-microsoft-azure-vms but most steps are also applicable for traditional Failover Cluster.
    Monday, June 6, 2016 8:32 AM

  • By the way, there is a PAYG StarWind Virtual SAN available in Azure Marketplace. 

    https://azure.microsoft.com/en-us/marketplace/partners/starwind/starwindvirtualsan/

    The StarWind VMs will require 2 NICs so it's recommended to deploy them using the ARM JSON template:

    https://github.com/starwinddeploy/azure-sw-cluster

    The minimum recommended VM size is D3. As Azure storage performance may vary over time it's recommended to do several benchmarks on "hot" and "cold" systems to see lower and higher performance watermarks. Premium storage should be used if the client system requires continuous high performance & low latency.

     

    Monday, June 6, 2016 11:14 AM
  • If I have 100TB of data to serve to the end users I would need 100TB + 100TB in Azure? 

    Do you know what rate of data change it has been tested with in Azure?

    Thank you

    Wednesday, June 8, 2016 1:29 PM
  • Hi, i do not have such information. 

    Anyways you should not make it as a 100TB single virtual volume or drive. Split into multiple smaller LUNs/Drives/Volumes/whatever even such amount of space should be no issue to any software including StarWind. 

    It might be more complicated to manage obviously but still remains the only safe way to keep things stable.

    Thursday, June 9, 2016 2:56 PM
  • If I have 100TB of data to serve to the end users I would need 100TB + 100TB in Azure? 

    Do you know what rate of data change it has been tested with in Azure?

    Thank you

    The best way to find out is to Ask MS folks who help you with Azure onboarding process. 

    The backend data tiering in azure is a very complex process and I've seen a lot of environments requiring "pre-heating" after long idle periods. 

    I agree with Taras, test it at different scales and make sure you're doing at least 3-4 cycles with different sets of "idle time" i.e. you write the data set, leave it there for a weekend and then do the test on Monday morning. 

    Wednesday, June 15, 2016 10:00 AM