Azure Blob Storage - Download of large files stops prematurely

Question

Hi,

we currently have an issue downloading Azure blobs >60MB via the Microsoft Azure API - it usually stops at 60 MB and leaves an incomplete file. The respective code is used in an ASP.NET IHttpHandler implementation:

context.Response.Buffer = false;
context.Response.Charset = "UTF-8";
context.Response.AddHeader("Content-disposition", string.Concat("attachment; filename=", fileInfo.FileName));
context.Response.ContentType = GetContentType(fileInfo.FileName);
context.Response.AddHeader("Content-Length", fileInfo.FileSizeBytes.ToString(CultureInfo.InvariantCulture));
context.Response.Flush();

this.handler.Download(fileInfo.StorageInfo, context.Response.OutputStream);

The "Download" method is implemented in the following way:

    public FileDownloadResult Download(string storageInfo, Stream downloadStream)
    {
      try
      {
        var storageInfoObj = AzureStorageInfo.FromString(storageInfo);
        var container = this.blobClient.GetContainerReference(storageInfoObj.ContainerName);
        var blob = container.GetBlobReference(storageInfoObj.BlobName);
        blob.DownloadToStream(downloadStream);
        return FileDownloadResult.Success;
      }
      catch (Exception ex)
      {
        log.ErrorFormat(
          CultureInfo.InvariantCulture, 
          "Error downloading file from Azure BLOB storage (storage info:\"{0}\"). Ex: {1}",
          storageInfo,
          ex);
        return new FileDownloadResult { ErrorInfo = ex.ToString() };
      }
    }

What could be the reason for the interruption of the download process?

Answer 1

Two things are happening in this code:

1. You're downloading a blob from blob storage.
2. You're sending the contents down to a web client.

Have you tried to isolate those two and figure out which is the problem? For example, you could use blob.DownloadToFile(...) to see if you get the whole file. Going the other way, you could have your handler send a local file instead of pulling a stream from blob storage. Either one may give you a clearer picture as to where the problem lies.

Answer 2

At least we managed to download the whole file using an open-source tool using the same Azure API. 

In the end, I passed a custom timeout parameter to the DownloadToStream method (generous 24 hours ;)),

and it worked... strange - did not see that usage of the mentioned method in any example i found.

So we can exclude that the error is located on the ASP.NET side i think...

Answer 3

Hi - do you know what exception you were getting?  Timeout could definitely cause an issue here, and I can let you know how to fix that in the code if that's the exception you were seeing.

---

-Jeff