locked
Need advice on whether to use Axum or CCR/DSS

    Question

  • Hi,

    About a month ago we bought licenses for the CCR/DSS tookit which we want to use to add distributed and concurrency capabilty to services that we provide as part of a product we have. By pure chance I came across Axum and need some advice on what to do. I have read your reply to Christian regarding the CCR origins but need some advice on what to use for our development? All our services are around processing a data queue (stored in a SQL table)  and running various steps against the entries. These steps are sometimes extremely quick (running xslt) or relatively slow (slow comapred to say running the xslt e.g. running data against external rendering server). At the moment, a single service runs on a box processing but we want to be able to have several services distributed processing these requests.

    My question is, should we also look at Axum to solve these problems - Is it too early to think about using it in a product and stick to the CCR/DSS or is this going to be around in the future so worth some investment? Were only a small company so I want to make the right decision and not have to refactor if there is a better option.

    Any advice would be gratefully received

    Kind regards

    Mike
    Thursday, May 21, 2009 10:32 AM

Answers

  • Mike,

    I would solve it in Axum using a pipeline:

    var startNode = new OrderedInteractionPoint<Record>();
    startNode ==> RunStandardNETComponent ==> StoreResultBackToDatabase ==> RunXSLT ==> MakeCallToExternalWCFService ==> WriteFinalResultToDB;

    And then start sending records to the startNode.

    If the nodes of the network are functional (the methods declared with the 'function' keyword), then the different stages of the network can execute concurrently, but preserving the message order.

    You can also enable additional opportunities for parallelism by creating multiple worker pipelines and load-balancing the work between them:

    var startNode = new OrderedInteractionPoint<Record>();
    var workers = new IInteractionTarget<Record>[4];
    foreach( var worker in workers )
        worker = RunStandardNETComponent ==> StoreResultBackToDatabase ==> RunXSLT ==> MakeCallToExternalWCFService ==> WriteFinalResultToDB;

    startNode -<: worker; // alternate operator

    This way the new elements arriving at startNode will round-robin through the worker pipelines.

    BTW, we have this explained in our Programmer's Guide and on our blog in more detail, if you're interested.

    Hope this helps!


    Artur Laksberg - MSFT
    • Marked as answer by Josh Phillips Saturday, May 23, 2009 2:14 AM
    Friday, May 22, 2009 8:26 PM
    Moderator

All replies

  • Hi Mike,

    Since Axum is an incubation project and we have not decided whether or when to ship it as a product, we do not recommend using Axum in a production environment or mission-critical applications.

    That said, based on your description, Axum should be a good fit technologically, and we would still be happy to hear your feedback. Are there any missing features, things you wish you could do but cannot, or simply things that don't work for you.

    Thanks,


    Artur Laksberg - MSFT
    Thursday, May 21, 2009 3:09 PM
    Moderator
  • Hi Artur,

    Thanks for ther reply - I guess I was expecting that response!

    Can I just then ask for your opinion. At the moment, we need to process lots of records in a database table (Queue A). Each record in Queue A must go through a serial process which at the moment looks like this pseudo code

    SerialMethod()
    {
    string RunStandardNETComponent(Record n) {return Output A) - Short Execution
    StoreResultBackToDatabase - Longer Execution
    RunXSLT(Output A){return Output B) - Short Execution
    int MakeCallToExternalWCFService - Much Longer Execution
    WriteFinalResultToDB

    }

    Obviously, the whole SerialMethod takes time to execute. At the moment we just read in 'x' number of records from Queue A and asynchrounously call SerialMethod with each one.
    This all runs in a Windows Service. I now want this to be able to run on many NT services to scale out the processing of Queue A (lets say the external calls are not a bottleneck).

    What do you think would be the best way of doing this in Axum?

    Would there actually be any merit in splitting up the functions of SerialMethod (agents I guess in Axum) even though each stage is dependant on the previous one?

    Any advice would be gratefully accepted

    Kind regards

    Mike
    Friday, May 22, 2009 3:28 PM
  • Mike,

    I would solve it in Axum using a pipeline:

    var startNode = new OrderedInteractionPoint<Record>();
    startNode ==> RunStandardNETComponent ==> StoreResultBackToDatabase ==> RunXSLT ==> MakeCallToExternalWCFService ==> WriteFinalResultToDB;

    And then start sending records to the startNode.

    If the nodes of the network are functional (the methods declared with the 'function' keyword), then the different stages of the network can execute concurrently, but preserving the message order.

    You can also enable additional opportunities for parallelism by creating multiple worker pipelines and load-balancing the work between them:

    var startNode = new OrderedInteractionPoint<Record>();
    var workers = new IInteractionTarget<Record>[4];
    foreach( var worker in workers )
        worker = RunStandardNETComponent ==> StoreResultBackToDatabase ==> RunXSLT ==> MakeCallToExternalWCFService ==> WriteFinalResultToDB;

    startNode -<: worker; // alternate operator

    This way the new elements arriving at startNode will round-robin through the worker pipelines.

    BTW, we have this explained in our Programmer's Guide and on our blog in more detail, if you're interested.

    Hope this helps!


    Artur Laksberg - MSFT
    • Marked as answer by Josh Phillips Saturday, May 23, 2009 2:14 AM
    Friday, May 22, 2009 8:26 PM
    Moderator

  • It's comprehensive, Thanks for your instruction!
    Sunday, January 16, 2011 2:56 AM