none
Is it possible to manage a message consumption pace from MSMQ RRS feed

  • Question

  • Hi,

    I have MSMQ queue capturing a messages from outside vendor. My internal BizTalk application is reading (extracting) those messages from the queue using MSMQ adapter.

    Problem:

    My internal BizTalk application could be saturated at some point during a pick hours due to the high volume of the messages coming from the queue.

    What I want:

    I would like to manage a message consumption pace from the queue (MSMQ). Ideally, I would like to have it configurable, so I can set a comfortable for BizTalk interface queue consumption pace (for example 10 messages per 1 sec).

    Questions:

    Do I have to create a custom MSMQ adapter or some external custom component?

    Any C# code samples with similar functionality available?


    Alex Star, New York

    Friday, January 15, 2016 2:52 PM

Answers

  • The first way I would try to solve this by isolating the MSMQ Receive in a separate Host.

    Problems caused by flood scenarios are usually because the high volume receive is causing other operations to throttle in that same Host.

    By isolating the receive, the Orchestration Hosts should keep processing while the messages just queue up in the MessageBox.  This does often require some tuning but is almost always solvable.

    If all else fails, you should be able to get what you want with a Pipeline Component that Sleeps for x seconds.

    • Marked as answer by Angie Xu Monday, January 25, 2016 3:41 AM
    Friday, January 15, 2016 3:09 PM
  • Well! Why do you need a custom MSMQ adapter when Microsoft has provided you the feasibility of Optimizing Performance of the MSMQ Adapter.

    On the receive side, you can have the adapter use a single execution thread. Whether the adapter uses a single thread or multiple threads depends on the setting of the Ordered Processing property on the receive location, as follows:

     

    • When the property is True, the adapter operates on a single thread. This limits the adapter to one message at a time and conserves memory. Notice that this effectively sets Batch Size to one (1), regardless of the value assigned to it in the property sheet.

     

    • When Ordered Processing is False, the adapter runs multiple threads and can process multiple messages at a time, therefore increasing performance.

     

    You must set Ordered Processing to True if you put a premium on managing server resources, or if the number or size of messages might exhaust available memory.

     

    • The MSMQ adapter submits messages to the MessageBox database in batches. The default batch size is 20, and the minimum batch size is 1. You can also control memory use by reducing the value of Batch Size on the receive location. A smaller batch size keeps fewer messages in memory and therefore uses less memory.

     

    • Placing send ports and receive locations on separate computers can also reduce memory use.

     

    You can also refer: How to Manage Multiple Receive Locations Using the MSMQ Adapter

    Saturday, January 16, 2016 8:17 PM
    Moderator

All replies

  • The first way I would try to solve this by isolating the MSMQ Receive in a separate Host.

    Problems caused by flood scenarios are usually because the high volume receive is causing other operations to throttle in that same Host.

    By isolating the receive, the Orchestration Hosts should keep processing while the messages just queue up in the MessageBox.  This does often require some tuning but is almost always solvable.

    If all else fails, you should be able to get what you want with a Pipeline Component that Sleeps for x seconds.

    • Marked as answer by Angie Xu Monday, January 25, 2016 3:41 AM
    Friday, January 15, 2016 3:09 PM
  • Well! Why do you need a custom MSMQ adapter when Microsoft has provided you the feasibility of Optimizing Performance of the MSMQ Adapter.

    On the receive side, you can have the adapter use a single execution thread. Whether the adapter uses a single thread or multiple threads depends on the setting of the Ordered Processing property on the receive location, as follows:

     

    • When the property is True, the adapter operates on a single thread. This limits the adapter to one message at a time and conserves memory. Notice that this effectively sets Batch Size to one (1), regardless of the value assigned to it in the property sheet.

     

    • When Ordered Processing is False, the adapter runs multiple threads and can process multiple messages at a time, therefore increasing performance.

     

    You must set Ordered Processing to True if you put a premium on managing server resources, or if the number or size of messages might exhaust available memory.

     

    • The MSMQ adapter submits messages to the MessageBox database in batches. The default batch size is 20, and the minimum batch size is 1. You can also control memory use by reducing the value of Batch Size on the receive location. A smaller batch size keeps fewer messages in memory and therefore uses less memory.

     

    • Placing send ports and receive locations on separate computers can also reduce memory use.

     

    You can also refer: How to Manage Multiple Receive Locations Using the MSMQ Adapter

    Saturday, January 16, 2016 8:17 PM
    Moderator