locked
SqlWorkflowInstanceStore Timeout RRS feed

  • Question

  • Our application hosts multple workflows and also multiple versions of each workflow. Each Version of every Workflow uses its own WorkflowSerivceHost and each of them has its own SqlWorkflowInstanceStore. The Instances of all Workflow Definitions are saved in the same Database.

    At one of our customers we have the problem that sometimes in (1/10 cases) when we call a recive activity from a client we get a timeout. After we recived the first Timeout all further calls to this workflow instance result also in a timeout.

    Today I could have reproduced the issue the first time in visual studio after I played around with the properties of the workflows SqlWorkflowInstanceStore. I noticed that my AppDomain.UnhandledException gets a async callback exception after the Timeout occures:

    Async Callback threw an exception.
    Type: System.Runtime.CallbackException
    Source: System.Runtime.DurableInstancing
    StackTrace:    at System.Runtime.AsyncResult.Complete(Boolean completedSynchronously)
       at System.Runtime.AsyncResult.AsyncCompletionWrapperCallback(IAsyncResult result)
       at System.Runtime.Fx.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result)
       at System.Runtime.AsyncResult.Complete(Boolean completedSynchronously)
       at System.Runtime.AsyncResult.AsyncCompletionWrapperCallback(IAsyncResult result)
       at System.Runtime.Fx.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result)
       at System.Runtime.AsyncResult.Complete(Boolean completedSynchronously)
       at System.Runtime.AsyncResult.AsyncCompletionWrapperCallback(IAsyncResult result)
       at System.Runtime.Fx.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result)
       at System.Runtime.AsyncResult.Complete(Boolean completedSynchronously)
       at System.Runtime.AsyncResult.AsyncCompletionWrapperCallback(IAsyncResult result)
       at System.Runtime.Fx.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result)
       at System.Runtime.AsyncResult.Complete(Boolean completedSynchronously)
       at System.Runtime.AsyncResult.AsyncCompletionWrapperCallback(IAsyncResult result)
       at System.Runtime.Fx.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result)
       at System.Runtime.AsyncResult.Complete(Boolean completedSynchronously)
       at System.Runtime.AsyncResult.AsyncCompletionWrapperCallback(IAsyncResult result)
       at System.Runtime.Fx.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result)
       at System.Data.Common.DbAsyncResult.AsyncCallback_Context(Object state)
       at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx)
       at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
       at System.Data.Common.DbAsyncResult.ExecuteCallback(Object asyncResult)
       at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(Object state)
       at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx)
       at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
       at System.Threading.ThreadPoolWorkQueue.Dispatch()
       at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()

    Workflow load failed. The given activity tree is not compatible with this instance.
    Type: System.InvalidOperationException
    Source: System.Runtime.DurableInstancing
    StackTrace:    at System.Runtime.AsyncResult.End[TAsyncResult](IAsyncResult result)
       at System.ServiceModel.Activities.Dispatcher.DurableInstanceManager.WaitAndHandleStoreEventsCallback(IAsyncResult result)
       at System.Runtime.Fx.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result)
       at System.Runtime.AsyncResult.Complete(Boolean completedSynchronously)

    I found a thread where someone has a simular problem:

    http://social.msdn.microsoft.com/Forums/en-US/wfprerelease/thread/d9b15f1e-2fc6-4fb4-bc64-cf8341917d06

    Also I noticed that in the RunnableInstancesTable the failing workflow instance (and the other workflows instances) have as RunnableTime the value 2986-10-27 17:10:56.133

    I think the problem is that I have mutiple service hosts which store their workflows in the same Database. When now one WorkflowServiceHost is called he tries to get the workflow instance but the workflow definition is locked by another WorkflowServiceHost (Currently I don't know why but it is locked.

    My question is, when there are Multiple WorkflowServiceHosts which go on one database how is decided which Workflow Instance belongs to which WorkflowDefinition? And what can I change that each WorkflowServiceHost loads only its own workflow instances?

    Also isn't it a kind of a bug of WF4 that I get on app domain level a unhandled AsyncCallback Exception and just a Timeout Exception on the client?
    Thursday, February 23, 2012 12:16 PM

All replies

  • Re: Exceptions since instance store operations occur async they are occuring on a different thread than the one that the client is invoking.  Exceptions never cross threads and therefore the AppDomain exception handler is invoked. 

    Re: Multiple WorkflowServiceHosts I don't believe there is a way to request that your host load only the instances it created.  Normally you want multiple hosts to load instances to support a web farm scenario.


    Sr. Program Manager, Windows Azure Platform Team(WF) http://blogs.msdn.com/rjacobs http://www.twitter.com/ronljacobs

    Thursday, February 23, 2012 7:30 PM
  • And how does the IIS solve this multiple ServiceHost Problem? I Thought the IIS could also host different WorkflowServices and has just one database?
    • Edited by Desert_Fox Friday, February 24, 2012 9:01 AM
    Friday, February 24, 2012 8:30 AM
  • Hi there,

    I have a very similar problem, where multiple versions of the same workflow of the same workflow share a persistence database. When a new version of the workflow is started (In its own Workflowservicehost, with a different base address) it tries to hydrate persisted instances of the old workflow and fails with that dreaded "System.Runtime.CallbackException", because the definition of the workflow has changed. So the route exception I get is something like:

    System.Runtime.CallbackException: Async Callback threw an exception. ---> System.Activities.ValidationException: 'VisualBasicValue<String>' is not of type 'Sequence'. When loading this instance you must ensure that the activity with name 'VisualBasicValue<String>' implements 'Sequence'.

    Is there really no way around this behavior? Is there not a namespace value (Or something similar) that I can set on the WorkflowServiceHost that will enable it to distinguish between its own instances in the persistence database and those belonging to other versions?

    If not is the only other option for me to automatically rename the workflow to have a version suffix ie. MyWorkflowV29 before a new workflowservicehost is created? Will this even help?

    Thanks for your time,

    Rob

    Thursday, March 1, 2012 5:21 AM
  • In the WF4 Database there is in the instances table the column WorkflowHostType in the InstancesTable, is this column for saving diffent workflow types?
    is there any way to set it in a WorkflowServiceHost?
    Friday, March 30, 2012 3:10 PM