locked
Slow AzCopy from Azure Files to a VM with Azure Premium Storage RRS feed

  • Question

  • Hi there, I am experiencing slow data transfers when trying to migrate 645gb worth of data from Azure Files to a VM in Azure that has a premium SSD drive.  I am migrating because of the very slow transfer rates we're getting from Azure Files.  I originally tried to do an Rsync and was only getting 2.5MB/s.  I then read up on AzCopy and was happy to see it was transferring at 30MB/s, but this throughput was very short lived.  After about 10 minutes, I noticed it was declining to the around 25MB/s, and then slowly started creeping down lower and lower.  I'm currently seeing 9.28 MB/s and it is continuing to lower.

    Again, this is all within Azure's environment, and in the same data center.  This migration is going to take forever if it continues to decline like this.  Is there a better way?

    Monday, January 21, 2019 5:36 PM

All replies

  • Have you considered configuring a service endpoint between the subnet where the VM resides and the storage account hosting the file share? hth Marcin
    Monday, January 21, 2019 5:49 PM
  • Thanks Marcin, no, I did not try that.  Do you think this would result in better sustained transfer rates?
    Monday, January 21, 2019 5:53 PM
  • I'd expect so. As per https://docs.microsoft.com/en-us/azure/virtual-network/virtual-networks-faq

    "Service endpoints add a system route which takes precedence over BGP routes and provide optimum routing for the service endpoint traffic. Service endpoints always take service traffic directly from your virtual network to the service on the Microsoft Azure backbone network. "

    hth
    Marcin

    Monday, January 21, 2019 6:01 PM
  • OK I tried this and I do not see a difference.  I stopped the AzCopy and retried it after the service endpoint was added, and am experiencing the exact same issue.
    Monday, January 21, 2019 6:37 PM
  • Just to update, I've tried a few things, to no avail.  I read somewhere that it could be the VM that is pulling the files being throttled by its iOPS limit.  I increased the VM size to a 4 core / 16gb machine, and while the initial file copy was noticeably faster (goes up to over 70MB/s reported by azcopy), after a few minutes it begins to drop off like before.

    I need to get our files off Azure Files and into the new managed Premium disk in the VM, and I can't.  I'd appreciate some help from MS here.  We run a SaaS service, so files are being generated all of the time, I'll need to do an initial sync of files, then a final one when we're ready to change over.  


    • Edited by Sharbel Monday, January 21, 2019 10:31 PM
    Monday, January 21, 2019 10:18 PM
  • @Sharbel Apologies for the delay!  Are you still facing performance issue? Kindly let us know if you still need any assistance on this issue.

    To transfer large numbers of files to Azure Files, we recommend that you use AzCopy  or Azure PowerShell. These tools have been optimized for network transfer.

    Thursday, February 14, 2019 11:14 AM