locked
Queue Message - dequeue behavior RRS feed

  • Question

  • Hi all,

    Tried to research how is the Azure Queue Storage react on dequeue, however not much documentation done by Microsoft on this. Could any one share that any documents / comment on:

    1) Is dequeue means deleting the message permanently OR make the message invisible for certain time?

    2) If making it invisible for certain time, how long & default timeout? How to change this parameter?

    3) Notice that there's a property called DequeueCount, so dequeue is done by my application OR storage account? 

    I read https://docs.microsoft.com/en-us/azure/storage/queues/storage-python-how-to-use-queue-storage. What I want to achieve is that I can delete the message and it never retry again cause it's no longer need.

    Thanks!

    Wednesday, January 16, 2019 2:14 AM

Answers

  • Is dequeue means deleting the message permanently OR make the message invisible for certain time?

    It's taking the message invisible and Deleting is handled by app post processing the message

    2) If making it invisible for certain time, how long & default timeout? How to change this parameter? 30 seconds is default.  You may refer to the suggestion mentioned in the SO link and this article

    3) Notice that there's a property called DequeueCount, so dequeue is done by my application OR storage account? 

    This count is maintained by Azure Queue service, this is per message count. The DequeueCount element has a value of 1 the first time the message is dequeued. This value is incremented each time the message is subsequently dequeued.

    Note: The DequeueCount element is returned in the response body only if the queue was created with version 2009-09-19 of the Queue service. For more info refer here: https://docs.microsoft.com/en-us/rest/api/storageservices/get-messages

    Refer here for information on CloudQueueMessage.DequeueCount Property

    What I want to achieve is that I can delete the message and it never retry again cause it's no longer need?

    As per your question I understand that your message takes 1 min to process, then the dequeue for 90 seconds will put it in invisibility mode and will give enough time to process and delete the message am I correct?

    maxDequeueCount	5	The number of times to try processing a message before moving it to the poison queue.

    Additional information:

    Please notice the DequeueCount property of the message, stating that it was de-queued one time. Now if we perform a peek to get all messages we just see the second message added because it was not de-queued. You can verify because the message ID is different as well.

    Storage Queue Trigger Samples

    You can get the number of times a message has been picked up for processing by adding an int parameter named dequeueCount to your function. You can then check the dequeue count in function code and perform your own poison message handling when the number exceeds a threshold.

    When you dequeue a message (GET Message in Azure Storage lingo), the message becomes invisible for certain amount of time and if not deleted by the process which dequeued it will become visible again and can be picked up by another process.

    The dequeue count is pushed as 'binding data'. (Each trigger has its own unique binding data specific to that trigger.) In C#, binding data can be directly bound to parameters, which is why the above example works. In Powershell, the binding data is passed via environment variables. Inspect your env variables and you should see these. (there's a nice example of this with HTTP here: https://blogs.technet.microsoft.com/stefan_stranger/2017/01/29/powershell-azure-functions-lessons-learned/ )


    Kindly let us know if the above helps or you need further assistance on this issue.

    Wednesday, January 16, 2019 10:19 AM

All replies

  • Is dequeue means deleting the message permanently OR make the message invisible for certain time?

    It's taking the message invisible and Deleting is handled by app post processing the message

    2) If making it invisible for certain time, how long & default timeout? How to change this parameter? 30 seconds is default.  You may refer to the suggestion mentioned in the SO link and this article

    3) Notice that there's a property called DequeueCount, so dequeue is done by my application OR storage account? 

    This count is maintained by Azure Queue service, this is per message count. The DequeueCount element has a value of 1 the first time the message is dequeued. This value is incremented each time the message is subsequently dequeued.

    Note: The DequeueCount element is returned in the response body only if the queue was created with version 2009-09-19 of the Queue service. For more info refer here: https://docs.microsoft.com/en-us/rest/api/storageservices/get-messages

    Refer here for information on CloudQueueMessage.DequeueCount Property

    What I want to achieve is that I can delete the message and it never retry again cause it's no longer need?

    As per your question I understand that your message takes 1 min to process, then the dequeue for 90 seconds will put it in invisibility mode and will give enough time to process and delete the message am I correct?

    maxDequeueCount	5	The number of times to try processing a message before moving it to the poison queue.

    Additional information:

    Please notice the DequeueCount property of the message, stating that it was de-queued one time. Now if we perform a peek to get all messages we just see the second message added because it was not de-queued. You can verify because the message ID is different as well.

    Storage Queue Trigger Samples

    You can get the number of times a message has been picked up for processing by adding an int parameter named dequeueCount to your function. You can then check the dequeue count in function code and perform your own poison message handling when the number exceeds a threshold.

    When you dequeue a message (GET Message in Azure Storage lingo), the message becomes invisible for certain amount of time and if not deleted by the process which dequeued it will become visible again and can be picked up by another process.

    The dequeue count is pushed as 'binding data'. (Each trigger has its own unique binding data specific to that trigger.) In C#, binding data can be directly bound to parameters, which is why the above example works. In Powershell, the binding data is passed via environment variables. Inspect your env variables and you should see these. (there's a nice example of this with HTTP here: https://blogs.technet.microsoft.com/stefan_stranger/2017/01/29/powershell-azure-functions-lessons-learned/ )


    Kindly let us know if the above helps or you need further assistance on this issue.

    Wednesday, January 16, 2019 10:19 AM
  •  Just checking in to see if the above answer helped. If this answers your query, do click “Mark as Answer” and Up-Vote for the same. And, if you have any further query do let us know.

    Tuesday, February 5, 2019 9:28 AM
  • May I know can I delete one of the messages within the queue? Not dequeue but delete.
    Friday, March 22, 2019 9:18 AM