none
Changing a message's visbility timeout outside of host.json? RRS feed

  • Question

  • Hello,

    I'm trying to implement some message retry logic for a QueueTrigger Azure Function in Visual Studio.

    The logic is the following:
    - if message processing fails, retry it immediately
    - if message fails again, retry it in 5 minutes
    - if message fails once more, retry it in an hour, and if that fails again, move it to a failed queue

    I know I can do most of this with a message's dequeueCount (to determine how many times the message failed processing) and its visibilityTimeout properties.

    However, even if I update the message's (a CloudQueueMessage) via a CloudQueueClient, after the function is done running, Azure Functions will set that message's timeout to the value defined in host.json (which I can understand).

    Is there a way such that I can bypass this, either by modifying the visibility value of host.json or preventing the function from applying the visiblity value after the function exits?

    Thanks!

    Tuesday, December 5, 2017 1:29 AM

All replies

  • As you know, you can configure the visibilityTimeout and maxDequeueCount properties for queue triggers, as described here: https://github.com/Azure/azure-webjobs-sdk-script/wiki/host.json ("queues" configuration section). However there is only the single visibilityTimeout value - it doesn't allow you to use a progressive timeout like you described.

    However, if your function is a VS "precompiled function" using the attribute model, you might try using Function Filters. You could write a custom error filter that will be called on error, allowing you to inspect the queue message and set your own timeout. This would be called after we've done our default processing (setting the default timeout value) which you could then override. Haven't tried this myself, but should work. Let me know :)


    Mathew Charles [MSFT]




    Tuesday, December 5, 2017 6:29 PM
  • Thanks for replying!

    Perhaps I'm not finding the solution in the host.json reference, as my question is more specifically about changing these properties within the function's code. 

    For example:

            [FunctionName("QueueTriggerTest")]
            public static void Run(
                [QueueTrigger("test", Connection = "AzureWebJobsStorage")] CloudQueueMessage myQueueItem,
                TraceWriter log)
            {
                log.Info(
                    $"C# Queue trigger function processed: {myQueueItem}, dequeue count is : {myQueueItem.DequeueCount}");
    
                if (myQueueItem.DequeueCount >= 1)
                {
                    changeHostJsonVisibilityTimeout("00:00:10");
                }
                else if (myQueueItem.DequeueCount == 2)
                {
                    changeHostJsonVisibilityTimeout("00:00:30");
                }
                else if (myQueueItem.DequeueCount == 3)
                {
                    changeHostJsonVisibilityTimeout("00:05:00");
                }
                else
                {
                    MoveToPoisonQueue(myQueueItem);
                }
            }
    That way I can change/control the message's visibility without using host.json's default value that's applied automatically after the function exits.


    edit: Saw your edit, will look into function filters!
    • Edited by Major T Tuesday, December 5, 2017 6:55 PM
    Tuesday, December 5, 2017 6:46 PM
  • Hey Mathew,

    I tried using the error filter, but I'm not sure if I'm able to get the CloudQueueMessage from the FunctionExceptionContext. I ended up using a custom exception to pass the message to my filter.

    However, it seems the function still proceeds as normal post-filter actions. In my filter, I set the CloudQueueMessage visible timeout to 3 seconds, with my host.json defining a default visibility of 3 minutes. After the exception filter runs, the message is still set with a visibility of 3 minutes. 


    EDIT:

    After more Stackoverflow searching, an alternative way I could do this is to send the CloudQueueMessage right back into my queue, and setting the initialVisibilityDelay parameter there...the only problem is that I lose the dequeuedCount.

    Friday, December 8, 2017 11:55 PM