none
Max Messaeges/size in BTS Batching ODX RRS feed

  • Question

  • Hi,

     

    I would like to know whether there is a limitation on the number of messages that a batching orchestration can take or there is  any size specification for the Batching ODX.???

     

    This for Doing Batching in BTS 2006 R2.


    Thanks... SPN
    Monday, March 21, 2011 11:43 AM

Answers

  • OK, so first off if the server has a 32 bit copy of windows it will not be able to use more than 4GB RAM period (it won't even see the rest).  If windows is 64 bit this is not really a as bad a problem.  You cannot upgrade 32bit windows to 64bit, so you would need to install fresh.  Fortunately because you have two BizTalk servers you could just take one down, install, and stand it up (attaching it to the existing group even).  Then do the same for the other (don't forget to install your MSIs on the newly installed servers). 

     

    Once you have 64bit windows running your BizTalk Servers you can create as many Hosts and Host instances as you like.  I would probably, just as a base, make these:

    • Receive Host
    • Send Host
    • Orchestration Host
    • Tracking Host
    • Batching Orchestration Host

    You then assign each adapter to run in which host (or you can choose at the port level once you've added to the adapter handler).  I would assign all other orchestrations to use the Orchesration Host and only the Batching orchestration to use its own host.  This will give it the maximum amount of memory and threads available to process faster with less risk of an out of memory exception. 

    If you'd like more information my website: http://www.novaenterprisesystems.com has contact information on it and I can help you more offline if need be. 

    Kind Regards,

    -Dan

    P.S. Please mark as answer if this answers your questions.

    • Marked as answer by SPN Biztalk Monday, April 4, 2011 2:27 PM
    Thursday, March 31, 2011 4:40 PM

All replies

  • I would take into account that Biztalk has a problem with large messages. 20 megs I believe is where there is a problem and xml messagaes grow very fast.

    My recomendation is to set the number of messages based on the agreement with trade partners. Typically, 1k messages is what I have ran into as the preference. However, that changes based on your client so you would have to talk to them to make the decission.


    Please Indicate "Mark as Answer" if this Post has Answered the Question
    Monday, March 21, 2011 2:36 PM
  • Hi CArlos,

     

    Thanks for the update.Appreciate Your Help.

    I ran a 50 MB(Flat File ) file last night and it took me 2hrs and 15 mins on a UAT server with many BTS applictions.I would say its okay for my testing.But i Would like to know the actual break point where the BTS Batching ODX breaks.The Transaction Count in the Management databaseTable value is a BiGInt if we are batching based on transactions.WE can send as many as BigInt Transactions to the Batching ODX.But I perfectly need the size where tht threshold.LEt me know if you have done any POC on that.

     

     

    Thanks... SPN
    Tuesday, March 22, 2011 5:45 AM
  • Well, I don't think there is one.  BizTalk wont break, honestly, it wont; at least not like most software.  It will really REALLY slow down (throttle) but it will not crash, I don't believe no matter how high the usage is.  That said, by default BizTalk will throttle with 50K active messages in the message box (most people don't know about the default settings). 

     

    Your main issue with that orchestration would be with memory usage.  If you're running a 32bit environment that's going to be fairly low: 1-2GB.  The reason it is not as high as it sounds is that these messages are in the DOM at that point (possibly) and if they are flat file to XML is generally one order of magnitude increase and XML to DOM is another... and if it is DOM it may be loaded twice -first to receive second to aggregate - but again I have not studied that specific orchestration much lately (it is a big one).  Run out of memory and the host instance running the orchestration will cycle and the orchestration will restart at the last persistence point (last step to read or write from the message box).

     

    If you're on 64bit hardware just make sure the host instance for that orchestration is not set to 32 bit and you should be set.  Either way if you make sure that one orchestration (batching) runs in it's own host / instance the throttling and memory issues will be less serious. 

     

    Kind Regards,

    -Dan

    Monday, March 28, 2011 10:43 PM
  • Thanks Dan,

     

    Appreciate you comments.I ran a 50 mb file with 2000 messages and 100 mb file with 4000 messages.The Batching did not break.But its consuming a considerable amount of CPU usage.It is pushing the boxes into RED zone.I have a 32  bit hardware with 24gb RAM with 1000 applications running at the same time.still i was able to get the output after dropping the 50 mb file.

     

    Need to consider the Landscape architecture once again before i proceed.

     

     


    Thanks... SPN
    Tuesday, March 29, 2011 5:36 AM
  • No problem, it's what I'm here for!

    When you say 32 bit hardware with 24GB RAM what exactly do you mean?  The maximum address space on 32bit is 4GB.  Do you mean 32bit BizTalk on Windows 64bit hardware?  Even with 32bit you can scale out in some ways, such as many hosts and host instances.  Can you give me more information about your environment?

    How many SQL Servers

    How many BizTalk Servers

    Hardware (CPU # and type, memory) and OS for all servers

    Kind Regards,

    -Dan

    Wednesday, March 30, 2011 1:38 PM
  • Hi Dan,

     

    Thanks for the correction.Its my BAd about the desciption.

    Its not 32bit Hardware but its 32 bit Windows 2003 OS with 32 bit BizTalk.

     

    How many SQL Servers--------------------------------------------------------2(Clustered)

    How many BizTalk Servers-----------------------------------------------------2

    Hardware:-

    CPU # and type,----------------------------------------------------------------4*2 ,Notsure on Type

    memory-------------------------------------------------------------------------24 gigs

     and OS for all servers---------------------------------------------------------Windows Server 2003.

     

    Let me know if you need more details.


    Thanks... SPN
    Thursday, March 31, 2011 12:29 PM
  • OK, so first off if the server has a 32 bit copy of windows it will not be able to use more than 4GB RAM period (it won't even see the rest).  If windows is 64 bit this is not really a as bad a problem.  You cannot upgrade 32bit windows to 64bit, so you would need to install fresh.  Fortunately because you have two BizTalk servers you could just take one down, install, and stand it up (attaching it to the existing group even).  Then do the same for the other (don't forget to install your MSIs on the newly installed servers). 

     

    Once you have 64bit windows running your BizTalk Servers you can create as many Hosts and Host instances as you like.  I would probably, just as a base, make these:

    • Receive Host
    • Send Host
    • Orchestration Host
    • Tracking Host
    • Batching Orchestration Host

    You then assign each adapter to run in which host (or you can choose at the port level once you've added to the adapter handler).  I would assign all other orchestrations to use the Orchesration Host and only the Batching orchestration to use its own host.  This will give it the maximum amount of memory and threads available to process faster with less risk of an out of memory exception. 

    If you'd like more information my website: http://www.novaenterprisesystems.com has contact information on it and I can help you more offline if need be. 

    Kind Regards,

    -Dan

    P.S. Please mark as answer if this answers your questions.

    • Marked as answer by SPN Biztalk Monday, April 4, 2011 2:27 PM
    Thursday, March 31, 2011 4:40 PM