locked
Embarrasing and unreliabe code for creating blob storage RRS feed

  • Question

  • Many months ago, it was recommended that I use the code below (by azure support) to create blob containers.  First, it's embarrasingly ugly code, however, I can live with that.  Second, and more important, it seems to be giving me unreliable results.  Even though the container is being reported to be created, about 1 out of 10 times when I try and access the container it appears to not be there.

    Is there a more recent fix for this?  This is really an important issue to me since I am often creating and deleting containers.

    Thanks.

               CloudStorageAccount cloudStorageAccount = GetCloudStorageAccount(username, out storageURL,
                                                 out azureSyncError2);
                GeneralUtils.LogDebug(username, "CreateContainer:storageURL:" + storageURL + " azureSyncError2: " + azureSyncError2, 0);
    
                if (azureSyncError2 == SyncError.Success)
                {
                  CloudBlobClient blogStorage = cloudStorageAccount.CreateCloudBlobClient();
                  CloudBlobContainer myContainer = blogStorage.GetContainerReference(containerName);
    
                  for (int i = 1; i <= 10; i++)
                  {
                    var blobContainer = myContainer;
                    blobContainer.CreateIfNotExist();
                    var stream = new MemoryStream();
                    try
                    {
                      blobContainer.GetBlobReference("__testblob").UploadFromStream(stream);
                      blobContainer.GetBlobReference("__testblob").Delete();
    
                      var permissions = blobContainer.GetPermissions();
                      permissions.PublicAccess = BlobContainerPublicAccessType.Container;
                      blobContainer.SetPermissions(permissions);
                      syncError = SyncError.Success;
                      break;
                    }
                    catch (StorageClientException e)
                    {
                      Console.WriteLine(e.Message);
                      Console.WriteLine("Failed creating container, retrying in 30 seconds...");
    
                      // Wait 30 seconds and retry
                      System.Threading.Thread.Sleep(30000);
                      if (i == 10)
                      {
                        // Only retry 3 times
                        syncError = SyncError.AzureCanNotCreateContainer;
                      }
                    }
                  }
                }
                else
                {
                  syncError = azureSyncError2;
                }
              }
              else
              {
                syncError = SyncError.AzureInvalidContainerName;
              }
    

    Peter Kellner http://peterkellner.net Microsoft MVP • ASPInsider
    Wednesday, March 16, 2011 3:58 PM

Answers

  • The Storage API to delete the container are Asynchronous  means when a request is made to delete a container the API returns quickly without waiting to verify if container is deleted. This API send a request in Storage controller to delete this container. If container is small the API returns expected results as the container deletion is done quickly and users can create a same name container. However if container has large data it will take X amount of time to delete it so setting synchronous  call would be hard to maintain as deleting gigs of data will take very long time. So if users are deleting a container with large amount of data the only option is to create a new container with some other name besides the one which was just deleted. Waiting for the container to delete is not a suggestion as there is no way to determine in current API if a container is deleted completely.

     

    Windows Azure Storage team knows this issue and working on the best possible way on recreation of containers but in the meantime our recommendation is to create a container with a unique name to get around this problem.

    Some solutions I have seen being used are:

    ·        container names with date as the suffix (day/month etc.).

    ·        maintain a master table of container names being used


    bill boyce
    • Proposed as answer by billb08 - MSFT Tuesday, March 22, 2011 9:14 PM
    • Marked as answer by Peter Kellner Wednesday, March 23, 2011 11:39 PM
    Tuesday, March 22, 2011 9:14 PM

All replies

  • Hi Peter,

    Could you provide a little bit more context as to what you are trying to achieve?

    1. Are you running multiple instances? [This should be a given, but it's good to know all the factors at play]
    2. What is the reasoning behind rapid create/delete of containers?

    Thanks,

    Cory


    Cory Fowler Windows Azure MVP http://blog.syntaxc4.net
    Wednesday, March 16, 2011 4:22 PM
    Answerer
  • Peter,

    If I am not mistaken, this is from one of the questions you posted here a while back where you're getting into issues where you tried to delete the blob container and the storage service told you that the container is deleted but because the actual container deletion process is a background process an attempt to create a new container by the same name resulted in this error (and hence the 30 seconds delay).

    Thanks

    Gaurav

    Wednesday, March 16, 2011 5:46 PM
  • My use case is that I will potentially have thousands of users each creating and deleting containers.  I don't expect them to be thrashing on the same folder over and over, but I do expect to see situations where people will delete a container with many gigabytes of stuff in it, then expect to be able to recreate it with the same name.

    For these scenarios, I hope to have a reliable way to both delete the container and recreate it, or at least get 100% consistent errors and return codes.  It does seem crazy to me that Azure Storage somehow hangs on to the container name once I delete the container.  What would make them want to hang on to the name?  Isn't the name just some entry in a hash someplace? 

    My biggest concern is that I be able to get accurate return codes and not have to do unusual things like plant little files.


    Peter Kellner http://peterkellner.net Microsoft MVP • ASPInsider
    Thursday, March 17, 2011 5:28 AM
  • I'm suspicious that this is what is happening:

     

    1.  I create a container named container1

    2.  I add a gig of stuff

    3.  I delete container1

    4.  I create a new container called container1 (same name, and get a successful return)

    5.  I start using the container and then get an error saying "Container is in the middle of being deleted"

    I'm having trouble pinning down the scenario, but I'm suspicious the above is happening.


    Peter Kellner http://peterkellner.net Microsoft MVP • ASPInsider
    Thursday, March 17, 2011 6:52 AM
  • The Storage API to delete the container are Asynchronous  means when a request is made to delete a container the API returns quickly without waiting to verify if container is deleted. This API send a request in Storage controller to delete this container. If container is small the API returns expected results as the container deletion is done quickly and users can create a same name container. However if container has large data it will take X amount of time to delete it so setting synchronous  call would be hard to maintain as deleting gigs of data will take very long time. So if users are deleting a container with large amount of data the only option is to create a new container with some other name besides the one which was just deleted. Waiting for the container to delete is not a suggestion as there is no way to determine in current API if a container is deleted completely.

     

    Windows Azure Storage team knows this issue and working on the best possible way on recreation of containers but in the meantime our recommendation is to create a container with a unique name to get around this problem.

    Some solutions I have seen being used are:

    ·        container names with date as the suffix (day/month etc.).

    ·        maintain a master table of container names being used


    bill boyce
    • Proposed as answer by billb08 - MSFT Tuesday, March 22, 2011 9:14 PM
    • Marked as answer by Peter Kellner Wednesday, March 23, 2011 11:39 PM
    Tuesday, March 22, 2011 9:14 PM
  • Thanks Bill,

    I'm marking this as the answer because it sounds like what is happening. Now, I have to wonder if the Azure team does not know what a pointer is?  It's like they are treating the container name as a physical location of the actual data.

    Can you imagine if in sqlserver, you could only create a new table if it had been some unspecified amount of time after you dropped one with the same name, and it mattered how many rows were in it before you dropped it?

    Makes you wonder if they followed the same principles with  the rest of the platform architecture.  (IMHO).


    Peter Kellner http://peterkellner.net Microsoft MVP • ASPInsider
    Wednesday, March 23, 2011 11:43 PM