Storage Blob 5-10 minute delay


  • We are logging various events, including performance metrics, to a storage container/blob in JSON format.  The issue we're seeing is that it appears Azure writes in batches every 5-10 minutes.  This makes utilizing this data in a product such as Splunk difficult since we aren't seeing near-realtime data.  Is this intended behavior?  Is there a way around it?  
    miercuri, 11 iulie 2018 16:44

Toate mesajele

  • Yes, this is due to the fact that for Blob Trigger, the SDK must monitor Azure Storage Blob logs to determine when new blobs have been added. This limitation is detailed in this document. The 5-10-minute delay comes from blob storage itself - that's how often it updates its blob logs.

    The document I linked to above describes the recommended pattern of en-queuing a message when you write a new blob, if you require more immediate processing. This is the best option for you if you control the creation of the blob - you can enqueue the message at that time.

    To determine why your blob requests are slower, the first thing to do would be to determine if the slow performance is server side or client side. Fortunately Azure Storage makes this easy via the Storage Analytics (Windows Azure Storage Logging: Using Logs to Track Storage Requests) - just compare the End To End latency and the Server Latency. I suspect you will see one of two things:

    1. Low E2E and Low Server. This would indicate that either the request is getting delayed being sent from the client (ie. not enough worker threads), or your logging is providing incorrect data.
    2. High E2E and Low Server. This would indicate a problem on the client side in processing the request (not enough worker threads to process the Response, slow processing of the memory stream, etc).


    If this answer was helpful, click “Mark as Answer” or Up-Vote. To provide additional feedback on your forum experience, click here

    miercuri, 11 iulie 2018 18:44
  • @Frentic :

    Just checking in to see if the above suggestions helped or you need further assistance on this issue.

    joi, 12 iulie 2018 20:18