locked
Only the last file is uploaded RRS feed

  • Question

  • I uploding collection of files to blob as parallel (IFormFiles in C#).

    But all files except the last are uploded only at 64 Kb. The latest file is fully uploaded.

    Here is a code:

    public static async Task UploadBulkAsync(string storageConnectionString, string containerName, Dictionary<string, IFormFile> blobNameToFile)
            {
                var completed = 0;
                var semaphore = new SemaphoreSlim(AppGlobalSettings.BulkUploadAzureBlobThreadCount, AppGlobalSettings.BulkUploadAzureBlobThreadCount);
                var tasks = new List<Task>();
                
                var options = new BlobRequestOptions
                {
                    ParallelOperationThreadCount = 20
                };
                
                var blobContainer = await GetContainerAsync(storageConnectionString, containerName);
                await blobContainer.SetPermissionsAsync(new BlobContainerPermissions() { PublicAccess = BlobContainerPublicAccessType.Blob });
                
                foreach (var one in blobNameToFile)
                {
                    var name = one.Key;
                    var file = one.Value;
                    
                    var blockBlob = blobContainer.GetBlockBlobReference(name);
                    blockBlob.Properties.ContentType = file.ContentType;
                    
                    await semaphore.WaitAsync();
    
                    tasks.Add(blockBlob.UploadFromStreamAsync(file.OpenReadStream(), null, options, null).ContinueWith(t =>
                    {
                        semaphore.Release();
                        Interlocked.Increment(ref completed);
                    }));
                }
                
                await Task.WhenAll(tasks);
            }




    • Edited by sergiy101 Wednesday, May 27, 2020 3:15 PM
    Wednesday, May 27, 2020 12:08 PM

Answers

  • The problem was in the collection of IFormFile objects. You cannot open multiple IFormFile streams in the same request processing. They need to be processed sequentially.

    link: https://stackoverflow.com/questions/55674322/getting-the-inner-stream-position-has-changed-unexpectedly-in-aws-lambda
    • Marked as answer by sergiy101 Monday, June 1, 2020 7:10 AM
    Monday, June 1, 2020 7:10 AM

All replies

  • For better understanding: Can you elaborate more on the issue?

    what is contained in blocks variable here var blocks = GetFileBlocks(file);? 


    Friday, May 29, 2020 6:31 AM
  • The problem was in the collection of IFormFile objects. You cannot open multiple IFormFile streams in the same request processing. They need to be processed sequentially.

    link: https://stackoverflow.com/questions/55674322/getting-the-inner-stream-position-has-changed-unexpectedly-in-aws-lambda
    • Marked as answer by sergiy101 Monday, June 1, 2020 7:10 AM
    Monday, June 1, 2020 7:10 AM