locked
Sequential Convoy and Long running transaction: Messages still referenced RRS feed

  • Question

  • Hi everyone<o:p></o:p>

    Being a BizTalk developer since 2006, this thing still stumps me.<o:p></o:p>

    I have a sequential convoy singleton orchestration that debatches messages using a rcvPipeline. The orchestration is needed in a FIFO scenario. In order to execute a rcvPipeline within an orchestration I need to encapsulate it within an atomic transaction scope. <o:p></o:p>

    In order to have an atomic scope the orchestration needs to be long running. I have also encapsulated the atomic transaction within a scope (using long running transactions) to have exception handling.

    <o:p></o:p>

    Everything works fine except for one major detail:
    When the orchestration executes the messages are still in the messagebox. I can even click on the orchestration instance in the MGMT console and look at the message! Tracking is disabled for the receive port as well as for the orchestration. Still, the messages does not get cleaned up.

    <o:p></o:p>

    I have set my DTA-purge to 1 hour and it works fine, but the messages are still in the orchestration.<o:p></o:p>

    My guess is that the long running transactions does not complete (although it looks like they should) and since the transaction is not completed the messages are not removed from the message box.

    So, to summarize: Is it possible to combine long running transactions and a singleton orchestration?


    //Mikael Sand (MCTS, ICC 2011) - Blog Logica Sweden

    Wednesday, November 12, 2014 2:32 PM

Answers

  • The messages are still being referenced by the orchestration! What can I do to make the atomic transation commit?

    In these cases, it usually the Orchestration the must Complete, not any specific Scope.  So long as the outer Loop doesn't exit, the Orchestration will never Complete.  The messages will remain references forever.

    Note, the inner Atomic Scope is completing, otherwise, the messages sent after debatching would never be committed to the MessageBox.

    • Marked as answer by Mikael Sand Thursday, November 13, 2014 7:39 PM
    Thursday, November 13, 2014 1:21 PM
    Moderator
  • This a known issue with Sequential convoy, thus Microsoft always recommend to have an exit condition in loop.

    Like every day at 12:00 AM your orchestration instance can terminate or it can get terminated after some configured delay.

    This will complete the orchestration and all messages will be de-refferenced. And later new incoming messages will initialize a new orchestration.


    Thanks,
    Prashant
    ----------------------------------------
    Please mark this post accordingly if it answers your query or is helpful.

    • Marked as answer by Mikael Sand Thursday, November 13, 2014 7:39 PM
    Thursday, November 13, 2014 12:48 PM

All replies

  • Yeah,

    but it because the atomic tx is not complete, and the messages are kept in the MessageBox till the tx commit. Did you try to include only the Transform and Send shapes into this atomic shape not the whole Loop? It you did this by design and the atomic transaction should include all debatched messages inside, this behaviour, when all messages are kept, is logical. 


    Leonid Ganeline [BizTalk MVP]

    Wednesday, November 12, 2014 5:13 PM
    Moderator
  • Hi Michel ,

    Yes we can have Singleton Orchestration inside Long running transaction no issues with it :)

    The problem here is the transaction which you committing inside Orchestration .

    The flow should be like this

    Thanks

    Abhishek

    Wednesday, November 12, 2014 5:39 PM
  • Sorry but this solution is not possible, since the pipeline used for debatching the messages has to be declared within an atomic scope and using your solution the pipeline variable cannot be accessed outside of the atomic scope (the looping part).

    I think this might connected to the fact that the orchestration is using a partner port. I will try just sending the messages out in a direct bound or physical port.


    //Mikael Sand (MCTS, ICC 2011) - Blog Logica Sweden

    Thursday, November 13, 2014 9:15 AM
  • I have to debatch using a pipeline and that can only be declared wothin an atopmic scope. In order to access the messages in the pipeline and send them, the pipeline must be available within the loop, and since the loop encapsulates the shapes, it must be within the scope.

    I was thinking about the Isolation Level for the atomic scope. It is set to Serializable. Do you think another setting might help?


    //Mikael Sand (MCTS, ICC 2011) - Blog Logica Sweden

    Thursday, November 13, 2014 9:28 AM
  • So after a day of looking for the solution it is quite clear that you are right in that the atomic transaction does not commit. I added a compensation block with trace info and it is never hit.

    I also experimented with Isolation level on the atomic transaction and that did nothing.

    Lastly I also made the sendport direct bound and also tried "specify later binding" to a physical port.

    The messages are still being referenced by the orchestration! What can I do to make the atomic transation commit?


    //Mikael Sand (MCTS, ICC 2011) - Blog Logica Sweden

    Thursday, November 13, 2014 12:11 PM
  • This a known issue with Sequential convoy, thus Microsoft always recommend to have an exit condition in loop.

    Like every day at 12:00 AM your orchestration instance can terminate or it can get terminated after some configured delay.

    This will complete the orchestration and all messages will be de-refferenced. And later new incoming messages will initialize a new orchestration.


    Thanks,
    Prashant
    ----------------------------------------
    Please mark this post accordingly if it answers your query or is helpful.

    • Marked as answer by Mikael Sand Thursday, November 13, 2014 7:39 PM
    Thursday, November 13, 2014 12:48 PM
  • The messages are still being referenced by the orchestration! What can I do to make the atomic transation commit?

    In these cases, it usually the Orchestration the must Complete, not any specific Scope.  So long as the outer Loop doesn't exit, the Orchestration will never Complete.  The messages will remain references forever.

    Note, the inner Atomic Scope is completing, otherwise, the messages sent after debatching would never be committed to the MessageBox.

    • Marked as answer by Mikael Sand Thursday, November 13, 2014 7:39 PM
    Thursday, November 13, 2014 1:21 PM
    Moderator
  • I totally agree to John's(Boatseller) comment and so till the orchestration is not terminated messages will be referenced in admin console.

    So as suggested try to have some exit condition where orchestration instance can terminate.

    Let us know if you have any further question.


    Thanks,
    Prashant
    ----------------------------------------
    Please mark this post accordingly if it answers your query or is helpful.

    Thursday, November 13, 2014 4:04 PM
  • The solution in the end was to simply not use the pipeline at all, but to instead use "XPATH-debatching". Using that I needed no transactions whatsoever :-) but as you all know; I gained alot of persistance points.

    Thanks for all the help.


    //Mikael Sand (MCTS, ICC 2011) - Blog Logica Sweden

    Thursday, November 13, 2014 9:30 PM
  • Hi Mikael,

    I am not sure if XPath Debatching is the best solution because using XPath can cause performance issues as complete message will be loaded in memory.

    I have seen many cases where it won't be able to clear the PERF Testing or have performance issues in Production.

    So, if you are using Xpath Debatching make sure to be sure about performance metrics.


    Thanks,
    Prashant
    ----------------------------------------
    Please mark this post accordingly if it answers your query or is helpful.

    Thursday, November 13, 2014 9:54 PM
  • You can also use the Loopback Adapter* to debatch in an external Pipeline: http://www.twoconnect.com/loopback-biztalk-adapter-free-download/

    *Disclaimer, shameless promotion of my own creation.

    Thursday, November 13, 2014 11:41 PM
    Moderator
  • Yes I know. That is why I mentioned the persistence points above. One upide of using an atomic transaction is that it forces onlöy one persistence point.I will deply and monitor processor usage ans well as DTA database size.


    //Mikael Sand (MCTS, ICC 2011) - Blog Logica Sweden

    Friday, November 14, 2014 8:04 AM
  • Nope! I cannot debatch in a pipeline since that does not garantee ordered delivery. If it did, I would not have to use an orchestration at all.

    //Mikael Sand (MCTS, ICC 2011) - Blog Logica Sweden

    Friday, November 14, 2014 8:07 AM