Delay activity firing multiple times in Statemachine

Question

Hi there,

I am having a problem with a delay activity ellapsing multiple times where it should only occur the once.

I have a statemachine workflow that has a 'Cancelable' state where a user has a configurable time-frame in which to either cancel the workflow process, or if they're impatient continue on with the process - once the time-span is exceeded the workflow should proceed automatically anyway.  This is what I have in my workflow:

Cancelable [StateActivity]

   |_ Proceed [EventDriven activity that houses a WCF Recieve activity]

   |_ Cancel [EventDriven activity that houses a WCF Recieve activity]

   |_ Timeout [EventDriven activity that houses a Delay activity]

Each event driven activity basically logs some debugging information before transitioning to an appropriate state.

What is happening is that the delay activity and related code is being invoked multiple times (mostly five times though once or twice it has reportedly done less).

Through debugging and logs I can see that each time the timer ellapses it does so on a new thread meaning that I get multiple instances of the same workflow (same Guid) running at once.

The problem (I think) seems to be something to do with the SqlPersistence service as when I set UnloadOnIdle="false" the problem goes away - however since this workflow can run for weeks this is not an option.

Some details:

* The workflow is self-hosted via WCF (pure app.config, no programatic hosting).

* The workflow 'implements' two WCF service contracts

* The workflow runtime has 5 services registered:

   * SqlWorkflowPersistenceService

   * ExternalDataExchangeService

   * SharedConnectionWorkflowCommitWorkBatchService

   * SqlTrackingService

   * and a custom logging service

* The workflow uses two customs ServiceBehaviours

   * One to register a custom IErrorHandler

   * One to manage NHibernate sessions between requests.

* The workflow is 1 of 5 workflows that are all configured in the same config file and are exposed via the one service - thought the have different endpoints.

* The workflow endpoints all use wsHttpContextBinding

Any help would be greatly appriciated.

Thanks in advance.

Answer 1

Okay I've isolated the cause of the problem but not the solution.  In my project I have five separate but interconnected workflows, each of which requires to be able to recieve external events via WCF - thus each are long-running workflows.

Five separate workflows means that I have to have five separate <service> sections in my app.config - since the service name has to be the concrete class that implements to service contract.  Each service uses the same behaviour configuration which registers the SqlPersistenceService as one of its workflowruntime services.  What is happening (I think) is that the SqlPersistenceService is being registered five times - once for each service - thus when the delay activity ellapses it is occuring five times.  I can support this claim by commenting out services in the app.config, when this is done the delay activity will only fire the same number of times as there are services.

My hunch is that each service is creating and running its own workflow runtime with all the associated wf-services attached, and since each runtime uses the same database connection it will poll the database five times, find the delay activity's expirey time five times, and notify the workflow five times.

I see two solutions to this problem:

(a) Use a separate database for each workflow.

(b) Some how get my services to use the same workflow runtime instance.

(b) is my prefered option but does anybody know how I could achieve this?

Answer 2

you are correct in your diagnosis of the problem.  You have five runtimes and five persistence services all sharing the same database, so each runtime is loading the workflow after the delay, and it is not surprising that you get different numbers each time. 

Also, the easiest fix is to use five different databases, or uniquely identify each service as an application as I showed in a recent MSDN article [1].

Since the workflow runtime is managed by the WorkflowRuntimeBehavior, and that is added in by the WorkflowServiceHost, you'll end up writing a lot of code to rebuild all of that to support the model you want.  To me this is one of the biggest failings of the WCF/WF integration in .NET 3.5. 

 

Matt

 

[1] http://msdn.microsoft.com/en-us/magazine/cc721606.aspx

Answer 3

Matt,

I dont think because of all five Runtimes are pointing to same Persistence store this is happening.

Because, If we see the Persistence database, we have owner_id and owned_untill fields.

This table contains the follwong columns

1. uidInstanceID – Workflow Instance ID will be stored.
2. state – The serialized state of the Workflow Instance
3. status – The Status of the Workflow instance
   1. 0 – Running/Executing
   2. 1 – Completed
   3. 2 – Suspended
   4. 3 – Terminated
   5. 4 -- Created
4. unlocked – Indicates whether the workflow Instance is owned by another runtime or not.
5. blocked – Indicates whether the workflow is waiting for an external event or not
6. info – Provides additional information when a workflow instance Terminate or Suspended.
7. modified – A DateTime field indicates when the State was last updated/modified
8. ownerID – The Current owner Guid of the Persistence service.
9. ownedUntil – The DateTime field indicates the ownership of the State object in the persistence store
10. nextTimer – will be populated by the Max value of DateTime value if the workflow is waiting for an external event other than Delay activity. If it is delay, the delay time will be added to the Current TimeStamp when the workflow was inserted into the persistence store.

If Runtime1 processing the workflow, owner_id and owned_untill will be filled.

Once the execution completes, wf instance will moved from persitence database.

Unless Runtime1 completes its execution, Runtimes 2 ot 5 should wait to acquire the ownership over the wf instance.

If it is so, there is no way where the same workflow can be executed 5 times because of the multiple runtimes.

 

 

Answer 4

Are you configuring the ownership timeout on the service behavior / workflow runtime?  If not, then the sql persistence service does not uniquely identify each instance of the service.  You can check the persistence database at runtime to see if the ownerid and owneduntil values are getting populated, they don't in all cases. 

 

matt

 

 

Answer 5

Thanks for your replies guys.  I tried setting the ownership timeouts previously but with no success.  For the mean time I've bodged in a 'solution' to my problem by changing the load intervals on each service to make it statistically unlikely (but not impossible) that multiple events will fire.

I think I'll end up moving to a programatically hosted solution for my workflow in the long run.