locked
Architectural guidance requested RRS feed

  • Question

  • Greetings.  I'd like some guidance on how to architect the system I'm about to describe.  There are so many options now and we're right at the cusp of Windows 2008 deployment so I'm looking for some direction.

    I want to track customer movement around the property (large casino/hotel/spa).  This does not need to be real-time but I need to log the activity event within a minute or so of its occurrence so I can notify interested parties.

    So, customer walks into a bar (or store, or spa, or front desk, etc.).  They are uniquely identified by a loyalty card.  Whichever system sees the event notifies a central service (probably web service due to the heterogeneous nature of our systems) that the event occurred.  This step needs to complete VERY quickly since it could block a customer-facing application.

    The event is stuck in a queue (MSMQ if using a queue is the right thing to do). 

    A listener de-queues the message and runs a series of operations (WF?) to identify the customer, notify interested parties based on customer status/type and log the activity for later mining.

    Seems pretty straight forward until you consider that on semi-busy days there could be hundreds of these events per minute (thousands on weekends).

    I'm leaning toward a WCF/WF-based solution but ran into a couple of problems/questions.

    1. Not sure how to best architect for the very high load situation.  Will .NET do the right thing as far as threading this or should I build a threaded host myself?

    2. WF and MSMQ don't play well together but maybe hosting in a particular way makes that a non-issue.

    3. Should I consider one of the public domain service buses for this (nServiceBus, SimpleServiceBus, etc.)?

    BizTalk is not an option since capital budgets have been all but eliminated. 

    Ideas and suggestions?

    Thanks in advance!


    Tuesday, February 3, 2009 2:30 PM

Answers

  • BizTalk would have been ideal, I think.

    MSMQ is good also, you're going to have to do a lot of the stuff the MSMQ does out of the box yourself.

    To be fair I wouldn't expect any of those technologies to be the bottleneck, I'd expect it to be the code that sits behind them that causes troubles.

    What I'm talking about is locking - if you're implementing any threads, and use use a thread pool, you will have to consider what impact that has.  My preference it toward asynchronous communicationas they're not blocking in any way.

    How complicated is the workflow for the customer operations?  Does it change often?  Is there any reason to need WF?  You could achieve the same sort of scenarios with a provider type of setup if things change, or have different scenarios.  I guess really it depends on how complex the operations are, as to how much of WF you're actually going to end up using.  If it's a significant amount, then it is a good choice.

    Is there a particular reason why you're chosing MSMQ and WCF?  MQ would achieve a similar result, and potentially you could distribute the queues for portions of the operations onto a number of machines.  There we're working toward being very similar to BizTalk though!

    You're also going to have to consider your infrastructure - what are the target environments like?  That is also going to lean you toward a particular technology choice. 

    Think that there's far too many things that affect the solution to give any real guidance, my one recommendation is don't pick this technology because it's cool.  It can be the right choice first, then if it's cool too, that's just a bonus!!

    Thanks,

    Martin.

    MCSD, MCTS. Please mark my post as helpful if you find the information good!
    Wednesday, February 4, 2009 11:07 AM

All replies

  • BizTalk would have been ideal, I think.

    MSMQ is good also, you're going to have to do a lot of the stuff the MSMQ does out of the box yourself.

    To be fair I wouldn't expect any of those technologies to be the bottleneck, I'd expect it to be the code that sits behind them that causes troubles.

    What I'm talking about is locking - if you're implementing any threads, and use use a thread pool, you will have to consider what impact that has.  My preference it toward asynchronous communicationas they're not blocking in any way.

    How complicated is the workflow for the customer operations?  Does it change often?  Is there any reason to need WF?  You could achieve the same sort of scenarios with a provider type of setup if things change, or have different scenarios.  I guess really it depends on how complex the operations are, as to how much of WF you're actually going to end up using.  If it's a significant amount, then it is a good choice.

    Is there a particular reason why you're chosing MSMQ and WCF?  MQ would achieve a similar result, and potentially you could distribute the queues for portions of the operations onto a number of machines.  There we're working toward being very similar to BizTalk though!

    You're also going to have to consider your infrastructure - what are the target environments like?  That is also going to lean you toward a particular technology choice. 

    Think that there's far too many things that affect the solution to give any real guidance, my one recommendation is don't pick this technology because it's cool.  It can be the right choice first, then if it's cool too, that's just a bonus!!

    Thanks,

    Martin.

    MCSD, MCTS. Please mark my post as helpful if you find the information good!
    Wednesday, February 4, 2009 11:07 AM
  • Thank you Martin.  Answering some of your questions:

    Our infrastructure is 95% Windows-based but the iSeries components generate most of the events. This solution will run on Windows platforms (Server 2003 or 2008).

    The workflow is not terribly complex.  WF might indeed be overkill which is why I asked about using it at all.  I'm really not sure it buys me much.  Basically it's a) identify the customer and load their profile; b) based on profile attributes notify (in various ways from email to paging) interested parties (from security to customer relations); c) record the activity (to SQL 2005 DB).  That's it for now.

    I was leaning toward the disconnected solution I describe to simplify what the iSeries developers (and other clients that report events) have to do (just invoke a method on a web service) and because if the workflow does become more complex I don't want that to slow down the reporting of events.

    I chose MSMQ so I don't lose events in the case of system availability and to provide the elasticity I need between reporting the event and acting on it as load varies.

    WCF not because it's cool (although it is) but because it seemed to offer more flexibility for future expansion of the service and works directly with MSMQ.  I was hoping that would simplify the development task on the listener side since it handles the low-level interaction with the queue.

    You stated your preference is toward asynch communications.  Are you familiar with nServiceBus and SimpleServiceBus?  Those are both asynch communication implementations that I may try to leverage here.  They also have the advantage (and associated pit falls) of being free.  Are they worth considering?

    Thanks!
    Wednesday, February 4, 2009 1:23 PM