none
Azure Data Movement Library upload speeds drop when uploading large files

    Question

  • Hi,

    I'm using Azure Data Movement Library (0.5.0) to upload large (2+ TB) files to Azure. I can normally achieve ~1Gbps upload speeds when uploading smaller files (in the 1-2GB range), but whenever I try to upload a TB sized file, my upload speeds drop to <100Mbps. Is there any special configuration I need to do to achieve equivalent upload speeds with GB size files and TB size files? 

    My upload code is pretty similar to this:

    ServicePointManager.DefaultConnectionLimit = Environment.ProcessorCount * 8;
    ServicePointManager.Expect100Continue = false;
    
    string source = ... // Path to 2TB file
    string destinationToken = ... // SAS Token
    CloudBlockBlob destination = new CloudBlockBlob(new Uri(destinationToken));
    
    await TransferManager.UploadAsync(source, blob);

    Any advice on uploading large files to Azure blob storage?

    Monday, January 30, 2017 10:42 PM

All replies