When does a HEAD request succeed for a BLOB being uploaded?

    General discussion

  • I am trying to identify we're able to verify an image exists in Azure storage via FetchAttributes, but then subsequently the file is missing from the container.  I can say, there is no process that is deleting content from the container... we are only uploading to it.

    After an image is uploaded, we peform a blob.FetchAttributes() to confirm it's existence in Azure and mark the result in the DB.  Sometimes, we notice images that are missing, but the database confirms the process marked it as being successfully loaded and verified in Azure.

    I believe it may be a concurrency issue with our application.  We have a windows service that polls a database table for a list of images to upload to Azure (sometimes many thousand at a time).  It loads those records into an in-memory queue, which X-number of worker threads have access to when processing the individual records.  If the queue is very long, subsequent polling of the DB for unprocessed records may result in duplicate records being appended to the queue.  Since the workers do not re-upload images if they already exist in Azure, this duplciation is tolerated.

    Because we see the images missing sporadically, I'm wondering what Azure would do if 2 separate threads were to upload the same blob (image file) at the same time.  Could there ever be a condition where a FetchAttributes would return successfully without having the blob actually committed to Azure?

    Obviously, we are working to resolve our concurrency issue to prevent the possibility of 2 threads working on the same file at the same time.  But right now, this is just a theory, and knowing how Azure handles a condition like this might give us confidence in any fix we implement.

    Thanks in advance for any response.


    Tuesday, January 31, 2012 2:58 AM

All replies

  • This should not happen. If a blob doesn't exist, a HEAD request should return 404. If later you send a HEAD request in a seperate application to those missing images, what do you get? If two threads try to create the same blob at the same time, only one should succeed.
    Wednesday, February 1, 2012 10:27 AM
  • I would tend to agree...  but what if the 2 threads start to upload the blob concurrently, and after the 1st thread completed, the 2nd thread subsequently failed.  Would this yeild a result where the blob exists in Azure for a brief moment, but then is removed from the container because the 2nd thread of execution did not complete it's upload?

    Wednesday, February 1, 2012 3:13 PM