none
Azure Service Bus Sessions - possible starvation RRS feed

  • Question

  • I have a background processor that listens to a service bus queue and messages need to be processed in order per Session ID.

    I'd like to ensure that no more than 5 messages are processed in parallel - meaning I'd like to process 5 sessions at once.

    If there are more than 5 sessions at any given time, is there any way I can ensure that all sessions are eventually handled?

    Even if I drop a certain session and then accept a new session after that, the same session might be processed again.  Can I ensure that the accepted session is the one that contains the oldest unprocessed message?

    Tuesday, May 21, 2019 2:25 PM

All replies

  • You can find the answer in Message sessions: first in, first out (FIFO)

    A new message can only be obtained when the prior message has been completed or dead-lettered. Abandoning a message causes the same message to be served again with the next receive operation.

    About parallel processing, you could have a pool of threads for processing, and every time you get a message, you just grab one of that pool and assign it a message. You need to manage that pool.

    OR, you could retrieve multiple messages at once and process them using TPL... for example, the method BeginReceiveBatch/EndReceiveBatch allows you to retrieve multiple "items" from Queue (Async) and then use "AsParallel" to convert the IEnumerable returned by the previous methods and process the messages in multiple threads.


    var messages = await Task.Factory.FromAsync<IEnumerable<BrokeredMessage>>(Client.BeginReceiveBatch(3, null, null), Client.EndReceiveBatch);
    
    messages.AsParallel().WithDegreeOfParallelism(3).ForAll(item =>
    {
        ProcessMessage(item);
    });

    That code retrieves 3 messages from queue and processes then in "3 threads" (Note: it is not guaranteed that it will use 3 threads, .NET will analyze the system resources and it will use up to 3 threads if necessary)

    Friday, May 24, 2019 4:04 AM
    Moderator
  • Thanks for the response.

    I understand that messages are processed in order by session, and I can receive messages in parallel.

    The case I'm worried about is not if there are more than 5 messages pending, but if there are more than 5 sessions at the same time.  I'd like to ensure that my worker eventually reads from every available session.  If I read 5 sessions in parallel, I might be listening on the same 5 sessions all the time, and never read from a newly created 6th session.

    Sunday, May 26, 2019 6:31 AM