Azure WebJob SDK ignores MaxPollingInterval RRS feed

  • Question

  • Greetings,

    I have an Azure WebJob SDK console application that I am running locally in Visual Studio so I can test it. 

    It runs jobs based on an storage queue.

    When I run the console application, it seems that it completely ignores config.Queues.MaxPollingInterval.

    I added the following code:

                Console.WriteLine($"MaxPollingInterval: {config.Queues.MaxPollingInterval}");

                Console.WriteLine($"IsDevelopment: {config.IsDevelopment}");

    and I get the following output:

    MaxPollingInterval: 00:03:20
    IsDevelopment: False

    However, the webjob application inventively de-queue the messages and runs all job without the delay I defined in  MaxPollingInterval.

    Why the job console application is not waiting for MaxPollingInterval reaches?

    Is this because I am running it locally in my dev machine? config.IsDevelopment is set to false.

    Thank you,

    Tuesday, April 11, 2017 4:05 AM

All replies

  • You can control the behavior of the MaxPollingInterval parameter by adding a few lines of code into your WebJob’s main method and update the desired values. For more information on this topic and the relevant code you can follow this blog post - Azure WebJobs & JobHostConfiguration.

    • Proposed as answer by Md Shihab Tuesday, April 11, 2017 6:34 PM
    • Unproposed as answer by amx2012 Tuesday, April 11, 2017 7:10 PM
    Tuesday, April 11, 2017 6:33 PM
  • Hello Md Shihab,

    Yes, I know how to change the value. That is not what I asked. Perhaps I did not explain it clearly.

    My question is the behavior of WebJobs SDK based on MaxPollingInterval.

    Even though I set MaxPollingInterval to 10 minute, the SDK intensively pulls all of the messages in Queue and try to process them. 

    If SDK processes all of the queue messages at once, without any delay defined in MaxPollingInterval, then what is MaxPollingInterval good  for?

    Thank you for help.

    Tuesday, April 11, 2017 7:16 PM
  • I'm definitely late to the conversation but I wanted to at least provide an answer.

    Firstly, consider the name's suffix: "Max". This is the longest it will wait. However, this says nothing about the shortest amount of time it will wait. That would be a "Min". That setting does not exist currently.

    The way it works by default (these numbers may not be accurate as I'm going by likely-faulty memory but you should get the point) is that it'll pull *up to* 16 messages off of the queue ("up to" means definitely 16 unless there just aren't that many messages there). Once 8 of the messages are done processing, it'll immediately pull *up to* 16 more. It'll keep doing this once it gets down to 8 remaining messages being processed until there just are no more messages.

    At this point, it'll then start to slow down in how quickly it polls the Queue for future messages. It may initially only wait 2 seconds, then 4 seconds, the 8 seconds, the 16 seconds, then 32 seconds, etc. until it hits your MaxPollingInterval period. I don't know the details of this "Back Off" policy but the source code should be available to explain it if you need more details.

    The moral of the story is that the MaxPollingInterval shouldn't be thought of as "the amount of time in between [anything]" but rather "the *longest* amount of time that a message will sit on a queue without being fetched after the queue has been idle for some extended period of time". This allows you to perform fast processing of messages while the queue is active but also slow down and save some money as the queue becomes idle (since hammering an idle queue does come with a financial expense).

    Monday, July 3, 2017 11:42 AM