locked
Speed up Azure File Service RRS feed

  • Question

  • Hello,

    We are playing around with Azure File Services. We wish to use it so that we can offload old static data to Azure.

    Is there a way in powershell to speed up the process to archive off data ?

    I have set the Cloud Tiering to cache files that were accessed or modified within 1 day.

    But if we want to perform the check sooner is there a why to push it via PowerShell ?

    Friday, December 6, 2019 4:15 AM

Answers

  • @Goce_D  Azure file services has high-end target performance. 
     
    To provide more insight on the performance query. There are two performance tiers for Files, standard and premium.

    Large file shares are GA on both standard and premium. For premium, all shares can go up to 100TB. But for standard, you must enable large file shares. Enabling large file shares enhances performance for standard with no increase in price.

    Standard files without the large file share feature enabled is 1000 IOPS and 60MB/s per share.

    Standard with large file share enabled will give double digit latency (typically high teens but can get higher). 10,000 IOPS & 300MB/s. These limits for standard are the same regardless of how large the file share is. You pay per transactions as well as per stored GB.

    Premium will give consistent low latency (~3ms), and IOPS and bandwidth according to the provisioned share size.

    For a 30TiB share, you would get 30,720 IOPS (30 * 1024) and be able to burst up to 92,160 IOPS (3x baseline). Throughput would be 1903MB/s read and 1268MB/s write. You pay for provisioned storage and transactions are included.

     To transfer large numbers of files to Azure Files, we recommend that you use AzCopy, Robocopy or Powershell These tools have been optimized for network transfer.

    The performance difference may because:  AzCopy is built against .Net Core framework which has better HTTP request handling speed than .Net Framework, if you are testing DataMovement Library on .Net Framework, the transfer speed will be a bit lower than it on .Net Core framework.

    Running DataMovement Library in .Net Core framework should be able to have a better performance.

    Note: Azure file performance metrics: Since the Azure File Sync agent runs on a Windows Server machine that connects to the Azure file shares, the effective sync performance depends upon a number of factors in your infrastructure: Windows Server and the underlying disk configuration, network bandwidth between the server and the Azure storage, file size, total dataset size, and the activity on the dataset. Since Azure File Sync works on the file level, the performance characteristics of an Azure File Sync-based solution is better measured in the number of objects (files and directories) processed per second.

    For Azure File Sync, performance is critical in two stages:

    Initial one-time provisioning: To optimize performance on initial provisioning, refer to Onboarding with Azure File Sync for the optimal deployment details.

    Ongoing sync: After the data is initially seeded in the Azure file shares, Azure File Sync keeps multiple endpoints in sync.

    To help you plan your deployment for each of the stages, below are the results observed during the internal testing on a system with a config

    *If cloud tiering is enabled, you are likely to observe better performance as only some of the file data is downloaded. Azure File Sync only downloads the data of cached files when they are changed on any of the endpoints. For any tiered or newly created files, the agent does not download the file data, and instead only syncs the namespace to all the server endpoints. The agent also supports partial downloads of tiered files as they are accessed by the user.

     Note: The numbers above are not an indication of the performance that you will experience. The actual performance will depend on multiple factors as outlined in the beginning of this section.

    See also: Troubleshoot Azure Files performance issues

    Hope this helps! 

    Kindly let us know if the above helps or you need further assistance on this issue.
    ------------------------------------------------------------------------------------------

    Do click on "Mark as Answer" and Upvote on the post that helps you, this can be beneficial to other community members.

    Friday, December 6, 2019 7:24 AM