none
What is efficient way to Debatch very large UnTyped Message in Orchestration RRS feed

  • Question

  • Hi,

    I wan to debatch untyped (System.Xml.XmlDocument) message of a very large size (approx 2 GB) by using orchestration. Could you please advise what can be the best possible approach? so that processing time and memory consumption should not be kept at minimum.

    Thank you

    Best Regards,
    Shahzad
     

    Wednesday, June 10, 2015 5:18 PM

Answers

All replies

  • Hi,

    Do not use a orchestration. Create a pipeline component to debatch and make the messages typed.

    Best Regards

    Rasmus Jaeger

    Thursday, June 11, 2015 6:49 AM
  • Agreed with Rasmus!

    Write a pipeline component using virtual streams in order to have good performance and avoid too much memory consumption.

    HTH,

    Sumit


    Sumit Verma - MCTS BizTalk 2006/2010 - Please indicate "Mark as Answer" or "Mark as Helpful" if this post has answered the question

    Thursday, June 11, 2015 6:56 AM
  • First, what is the likely dominant characteristic, number of messages or size of debatched messages?

    Then, why can't you debatch with the XmlDisassembler?  Meaning what measurable problem are you having with it?

    Thursday, June 11, 2015 11:08 AM
    Moderator
  • Reason why i cannot use XmlDisassember or any custom pipeline component is i want to send the complete message to party A and send de-batched messages to party B. If i debatched it on receive side then i won't be able to send complete message to party A
    Thursday, June 11, 2015 2:14 PM
  • No problem, you can execute Pipelines in Orchestrations:

    https://msdn.microsoft.com/en-us/library/aa562035.aspx?f=255&MSPPError=-2147217396

    You receive the file and route it to two places, directly to Party A and through a debatch Orchestration for Party B.

    Thursday, June 11, 2015 2:48 PM
    Moderator
  • Yes, you'd rather use a pipeline on receive port.

    Otherwise the whole message is published to the message box and bottlenecks it. You can imagine the SQL writing this huge message to the DB and then operates it as one piece. It is not efficient. Publishing all debatched messages is still not good, it will follow to the massive pipe processing and massive publishing spike and the MessageBox will be clogged.

    Can you just pass this large message by reference to BTS and not processing the message body in BTS at all?


    Leonid Ganeline [BizTalk MVP]

    Thursday, June 11, 2015 4:27 PM
    Moderator
  • Hi

    Use the XmlDisassember to debatch. Send the debatched messages to party B and batch the messages again to party A 

    Best regards

    Rasmus Jaeger

    Friday, June 12, 2015 7:30 AM
  • Hi Shahzad,

    As been suggested by Leonid ,You can have do a de batching at receive port level with custom receive pipeline component . If you are working with untyped xml document ,then i would suggest to use stream to load incoming messsage than have can divide the message into individual parts .

    Thanks

    Abhishek

    Monday, June 15, 2015 7:35 AM