locked
PersistableIdle event and WorkflowApplication.Load multithreading issues RRS feed

  • Question

  • Hello,

    I'm new to Workflow Foundation and I've encountered the following problems.

    Problem 1 - PersistableIdle event does not fire the Unload event when I return Unload after the call has finished

    I'm hosting my own Workflows and I'm not using the WorkflowServiceHost.

    I've got one workflow that I've created and everythings works as I expect in a single threaded environment.

    When I am processing more than one instance of the same workflow the PersistableIdle event completes and I return Unload but the Unload event never fires. I've created separate WorkflowApplication for each workflow instance, and each WorkflowApplication is sharing the same SqlWorkflowInstanceStore.

    Any ideas why this could happen, I don't get any errors?

    Problem 2 - WorkflowApplication.Load event hangs

    When running in a multi-threaded environment the thread hangs inside WorkflowApplication.Load method.

    Any ideas why this could happen, again I don't get any errors?


    • Edited by Din123 Wednesday, August 27, 2014 1:43 PM
    Wednesday, August 27, 2014 11:58 AM

All replies

  • put this in the web.config file and see if the logs helps.

     <system.diagnostics>
        <sources>
          <source name="System.Activities" switchValue="Information">
            <listeners>
              <add name="textListener" />
              <remove name="Default" />
            </listeners>
          </source>
        </sources>
        <sharedListeners>
          <add name="textListener"
               type="System.Diagnostics.TextWriterTraceListener"
               initializeData="MyTraceLog.txt"
               traceOutputOptions="ProcessId, DateTime" />
        </sharedListeners>
        <trace autoflush="true" indentsize="4">
          <listeners>
            <add name="textListener" />
          </listeners>
        </trace>
      </system.diagnostics>

    try putting Persist and load outside trabsactions (if any).

    using (var ts = new System.Transactions.TransactionScope(TransactionScopeOption.Suppress)) {
                  m_WorkflowApplication.Persist/Load();
                  ts.Complete();
                }

    PersistableIdle is called when the wf goes idle (for a boomark typically) ,, it is not called when you unload a wf or the wf end.

    HTH

    e.s.

    Wednesday, August 27, 2014 2:51 PM
  • Thanks for your questions.

    Problem 1

    Are you sure the workflow instance is not being aborted in your multi-threading environment? Do you provide a Action to the Aborted property of WorkflowApplication?

    Can you provide a sample of your code that handles the WorkflowApplication instances for loading and how your multi-threading works?

    Jim

    Wednesday, August 27, 2014 8:45 PM
  • I've enabled the logging and I see exactly the information I've logged in my log file. No new clues.

    After the PersistableIdle is completed (passing back Unload), the even Unload never gets called. As soon as I switch my code to a single thread it works.

    Thursday, August 28, 2014 7:55 AM
  • I'm 100% sure its not been aborted because I've put a log message in the abort event and can see it never gets written.

    I have 4 threads processing 4 different instances of the same workflow. They share the same SqlWorkflowInstanceStore.

    Creating WorkflowApplication

    var workflowApplication = inputData !=null?newWorkflowApplication(rootActivity, inputData) : newWorkflowApplication(rootActivity);

        

    Assigning values

    workflowApplication.InstanceStore = _instanceStore;
    

    Setting up InstanceStore, this is done when the service starts up.

    _instanceStore = new SqlWorkflowInstanceStore(ConfigurationManager.AppSettings["SqlWorkflowConnection"])
    {
        InstanceEncodingOption = InstanceEncodingOption.GZip,
        HostLockRenewalPeriod = new TimeSpan(00030),
        InstanceCompletionAction = InstanceCompletionAction.DeleteNothing,
        InstanceLockedExceptionAction = InstanceLockedExceptionAction.BasicRetry,
        RunnableInstancesDetectionPeriod = new TimeSpan(0005)
    };
    var instanceHandle = _instanceStore.CreateInstanceHandle();
     
    var createOwnerCmd = new CreateWorkflowOwnerCommand();
    var view = _instanceStore.Execute(instanceHandle, createOwnerCmd, TimeSpan.FromSeconds(30));
    instanceHandle.Free();
    _instanceStore.DefaultInstanceOwner = view.InstanceOwner;
    

    PersistableIdle event

    workflowApplication.PersistableIdle = e =>
    {
        var workflowDataAccess = DataAccessService.CreateWorkflowDataAccess(Modes.AutoAcknowledge);
        if (startWorkflow != null)
        {
            startWorkflow.ExecutionStatusId = (intExecutionStatuses.Complete;
            workflowDataAccess.AddUpdateStartWorkflowToDb(startWorkflow);
        }
     
        if (userRequestActionResponse != null)
        {
            userRequestActionResponse.ExecutionStatusId = (int)ExecutionStatuses.Complete;
            workflowDataAccess.AddUpdateUserRequestActionResponseToDb(userRequestActionResponse);
        }
     
        if (remoteProcedureCallResponse != null)
        {
            remoteProcedureCallResponse.ExecutionStatusId = (int)ExecutionStatuses.Complete;
            workflowDataAccess.AddUpdateRemoteProcedureCallResponseToDb(remoteProcedureCallResponse);
        }
        ETWLog.DebugFormat("PersistableIdle-> Persisted InstanceId {0}", e.InstanceId);
        return PersistableIdleAction.Unload;
    };
    

    Unload event

    workflowApplication.Unloaded =
        e =>
        {
            if (startWorkflow != null)
            {
                if (_workflowInstanceLocks.ContainsKey(e.InstanceId.ToString()))
                {
                    _workflowInstanceLocks[e.InstanceId.ToString()].Set();
                    ETWLog.DebugFormat("Unloaded-> WorkflowInstanceLocks {0} Set", e.InstanceId);
                }
            }
     
            ETWLog.DebugFormat("Unloaded-> Instance {0}", e.InstanceId);
        };

    After WorkflowApplication is created I do the following.

    ETWLog.DebugFormat("GetWorkflowApplicationByInstance-> {0} Loading Application", workflowInstanceId);
    application.Load(Guid.Parse(workflowInstanceId), TimeSpan.FromSeconds(5));
    ETWLog.DebugFormat("GetWorkflowApplicationByInstance-> {0} Loaded Application", workflowInstanceId);
    

    I create the WorkflowApplication object as shown above, the thread blocks before the application.Load is called otherwise there is nothing to load. Its waiting for the PersistableIdle event to persist the workflow instance and then unload the workflow which calls the ManualResetEvent.Set() method, only then the thread that created the WorkflowApplication tries to load the application.

    Also I've extended the TrackingParticipant class so I can add extra data to track.

    publicsealedclassDatabaseTrackingParticipant : TrackingParticipant     {         /// <summary>/// When implemented in a derived class, used to synchronously process the tracking record./// </summary>/// <param name="record">The generated tracking record.</param>/// <param name="timeout">The time period after which the provider aborts the attempt.</param>protectedoverridevoid Track(TrackingRecord record, TimeSpan timeout)         {             try             {                 if (record !=null)                 {                     if (record isWorkflowInstanceRecord)                     {                         var instanceRecord = record asWorkflowInstanceRecord;                         ETWLog.Info(string.Format("{0} InstanceId: {1} Workflow instance state : {2}",                             instanceRecord.ActivityDefinitionId, record.InstanceId, instanceRecord.State));

       

                                                     

    This does get called. Its the last code that gets called before it fails to call Unload. I'm also sure Abort event is not called as I've logged a message inside it.

    Here is the output from the log file.

    2014-08-28 07:48:39,586 [Async->07:48:39,603] [32] DEBUG Risk.Workflow.Service.WorkflowInstanceHost - WorkflowInstanceLocks 3379f9e5-1bf5-471b-8d4b-9538966918fe Added
    2014-08-28 07:48:39,666 [Async->07:48:39,671] [42] INFO Risk.Workflow.Service.DatabaseTrackingParticipant - DynamicActivity InstanceId: 3379f9e5-1bf5-471b-8d4b-9538966918fe Workflow instance state : Started
    2014-08-28 07:48:39,669 [Async->07:48:39,671] [42] INFO Risk.Workflow.Service.DatabaseTrackingParticipant - Activity: DynamicActivity WorkflowInstanceId: 3379f9e5-1bf5-471b-8d4b-9538966918fe ActivityInstanceId: 1 State : Executing
    2014-08-28 07:48:39,719 [Async->07:48:39,719] [42] INFO Risk.Workflow.Service.DatabaseTrackingParticipant - Activity: Flow Chart WorkflowInstanceId: 3379f9e5-1bf5-471b-8d4b-9538966918fe ActivityInstanceId: 1.1 State : Executing
    2014-08-28 07:48:39,770 [Async->07:48:39,771] [42] INFO Risk.Workflow.Service.DatabaseTrackingParticipant - Activity: WriteLine WorkflowInstanceId: 3379f9e5-1bf5-471b-8d4b-9538966918fe ActivityInstanceId: 1.123 State : Executing
    2014-08-28 07:48:39,775 [Async->07:48:39,776] [46] INFO Risk.Workflow.Service.DatabaseTrackingParticipant - Activity: WriteLine WorkflowInstanceId: 3379f9e5-1bf5-471b-8d4b-9538966918fe ActivityInstanceId: 1.123 State : Closed
    2014-08-28 07:48:39,779 [Async->07:48:39,782] [46] DEBUG Risk.Workflow.Service.Activities.ImportService.FileArrived - Step Name = FileArrived, InstanceId = 3379f9e5-1bf5-471b-8d4b-9538966918fe, Filename = C:\Temp\192660_20140821230157755.csv
    2014-08-28 07:48:39,779 [Async->07:48:39,782] [46] DEBUG Risk.Workflow.Service.Activities.RemoteProcedureCallActivity - Step Name = RemoteCodeExecution, InstanceId = 3379f9e5-1bf5-471b-8d4b-9538966918fe, RemoteCodeName = FileArrived
    2014-08-28 07:48:39,797 [Async->07:48:39,798] [46] INFO Risk.Workflow.Service.Activities.LogHelper - Calling RPC Code [FileArrived@Risk.DataSource.Import.Service] on WorkflowInstance [3379f9e5-1bf5-471b-8d4b-9538966918fe] with Input [[{"Key":"ProviderName","Value":"Portfolio"},{"Key":"Filename","Value":"C:\\Temp\\192660_20140821230157755.csv"}]].
    2014-08-28 07:48:39,799 [Async->07:48:39,800] [40] INFO Risk.Workflow.Service.DatabaseTrackingParticipant - Activity: File Arrived (Import Service) WorkflowInstanceId: 3379f9e5-1bf5-471b-8d4b-9538966918fe ActivityInstanceId: 1.114 State : Executing
    2014-08-28 07:48:39,801 [Async->07:48:39,802] [44] INFO Risk.Workflow.Service.DatabaseTrackingParticipant - DynamicActivity InstanceId: 3379f9e5-1bf5-471b-8d4b-9538966918fe Workflow instance state : Idle
    2014-08-28 07:48:39,803 [Async->07:48:39,804] [44] DEBUG Risk.Workflow.Service.WorkflowInstanceHost - Idle-> Instance 3379f9e5-1bf5-471b-8d4b-9538966918fe
    2014-08-28 07:48:39,809 [Async->07:48:39,810] [44] DEBUG System.Object - PersistableIdle-> Persisted InstanceId 3379f9e5-1bf5-471b-8d4b-9538966918fe
    2014-08-28 07:48:39,809 [Async->07:48:39,810] [15] INFO Risk.Workflow.Service.DatabaseTrackingParticipant - DynamicActivity InstanceId: 3379f9e5-1bf5-471b-8d4b-9538966918fe Workflow instance state : Unloaded
    2014-08-28 07:48:39,842 [Async->07:48:39,843] [0486043b-bb24-4f84-8101-b48f700fae07.Risk_Workflow_RemoteProcedureCallResponse.IncomingMessage] DEBUG Risk.Workflow.Service.Tasks.RemoteProcedureCallResponseTask - RemoteProcedureCallResponseTask-> Key 3379f9e5-1bf5-471b-8d4b-9538966918fe

    Also here is the log from the TraceWriter.

    System.Activities Information: 39457 : 39457Tracking Record WorkflowInstanceRecord { InstanceId = 3379f9e5-1bf5-471b-8d4b-9538966918fe, RecordNumber = 0, EventTime = 28/08/2014 07:48:39, ActivityDefinitionId = DynamicActivity, State = Started } raised to Risk.Workflow.Service.DatabaseTrackingParticipant.Risk.Workflow.Service.vshost.exe
    System.Activities Information: 39457 : 39457Tracking Record ActivityStateRecord { InstanceId = 3379f9e5-1bf5-471b-8d4b-9538966918fe, RecordNumber = 1, EventTime = 28/08/2014 07:48:39, Activity { Name=DynamicActivity, ActivityId = 1, ActivityInstanceId = 1, TypeName=System.Activities.DynamicActivity }, State = Executing } raised to Risk.Workflow.Service.DatabaseTrackingParticipant.Risk.Workflow.Service.vshost.exe
    System.Activities Information: 39457 : 39457Tracking Record ActivityStateRecord { InstanceId = 3379f9e5-1bf5-471b-8d4b-9538966918fe, RecordNumber = 2, EventTime = 28/08/2014 07:48:39, Activity { Name=Flow Chart, ActivityId = 1.1, ActivityInstanceId = 2, TypeName=System.Activities.Statements.Flowchart }, State = Executing } raised to Risk.Workflow.Service.DatabaseTrackingParticipant.Risk.Workflow.Service.vshost.exe
    System.Activities Information: 39457 : 39457Tracking Record ActivityStateRecord { InstanceId = 3379f9e5-1bf5-471b-8d4b-9538966918fe, RecordNumber = 3, EventTime = 28/08/2014 07:48:39, Activity { Name=WriteLine, ActivityId = 1.123, ActivityInstanceId = 3, TypeName=System.Activities.Statements.WriteLine }, State = Executing } raised to Risk.Workflow.Service.DatabaseTrackingParticipant.Risk.Workflow.Service.vshost.exe
    System.Activities Information: 39457 : 39457Tracking Record ActivityStateRecord { InstanceId = 3379f9e5-1bf5-471b-8d4b-9538966918fe, RecordNumber = 4, EventTime = 28/08/2014 07:48:39, Activity { Name=WriteLine, ActivityId = 1.123, ActivityInstanceId = 3, TypeName=System.Activities.Statements.WriteLine }, State = Closed } raised to Risk.Workflow.Service.DatabaseTrackingParticipant.Risk.Workflow.Service.vshost.exe
    System.Activities Information: 39457 : 39457Tracking Record ActivityStateRecord { InstanceId = 3379f9e5-1bf5-471b-8d4b-9538966918fe, RecordNumber = 5, EventTime = 28/08/2014 07:48:39, Activity { Name=File Arrived (Import Service), ActivityId = 1.114, ActivityInstanceId = 4, TypeName=Risk.Workflow.Service.Activities.ImportService.FileArrived }, State = Executing } raised to Risk.Workflow.Service.DatabaseTrackingParticipant.Risk.Workflow.Service.vshost.exe
    System.Activities Information: 39457 : 39457Tracking Record WorkflowInstanceRecord { InstanceId = 3379f9e5-1bf5-471b-8d4b-9538966918fe, RecordNumber = 6, EventTime = 28/08/2014 07:48:39, ActivityDefinitionId = DynamicActivity, State = Idle } raised to Risk.Workflow.Service.DatabaseTrackingParticipant.Risk.Workflow.Service.vshost.exe
    System.Activities Information: 1005 : 1005WorkflowApplication Id: '3379f9e5-1bf5-471b-8d4b-9538966918fe' went idle.Risk.Workflow.Service.vshost.exe
    System.Activities Information: 1041 : 1041WorkflowApplication Id: '3379f9e5-1bf5-471b-8d4b-9538966918fe' is idle and persistable.  The following action will be taken: Unload.Risk.Workflow.Service.vshost.exe
    System.Activities Information: 39457 : 39457Tracking Record WorkflowInstanceRecord { InstanceId = 3379f9e5-1bf5-471b-8d4b-9538966918fe, RecordNumber = 7, EventTime = 28/08/2014 07:48:39, ActivityDefinitionId = DynamicActivity, State = Unloaded } raised to Risk.Workflow.Service.DatabaseTrackingParticipant.Risk.Workflow.Service.vshost.exe

    For multi-threading I'm using the threads from ThreadPool to process each of the workflow instances.

    • Edited by Din123 Thursday, August 28, 2014 8:09 AM
    Thursday, August 28, 2014 8:02 AM
  • the load event hangs "forever" or it timeouts ?

    In my implementation i'm creating a different instance store for each workflowapplication , you could try to do that.

    Are you sure there is no ambient transaction active ? As I said I had many problems wiht ambient transactions (deadlocks timeouts and the DTC being invloved) until i did not wrap calls to persist and load in this way

    using (var ts = new System.Transactions.TransactionScope(TransactionScopeOption.Suppress)) {
                  m_WorkflowApplication.Persist/Load();
                  ts.Complete();
     }

    And i had to return PersistableIdleAction.none from the persistable idle event .. (i had no change to put TransactionScopeOption.Suppress around the implicit persistence that was done:  if i returned PersistableIdleAction.Persist i had an error about the DTC not nrunning in my machine.

    I've made my workflowmanager class implementing Idisposable , and i persist and unload the WF in the dispose method

    here is the code .. just in case

    using System;
    using System.Activities;
    using System.Activities.DurableInstancing;
    using System.Activities.Statements;
    using System.Activities.Tracking;
    using System.Activities.XamlIntegration;
    using System.Collections.Generic;
    using System.Data.SqlClient;
    using System.Diagnostics;
    using System.IO;
    using System.Linq;
    using System.Reflection;
    using System.Runtime.DurableInstancing;
    using System.Text;
    using System.Threading;
    using System.Threading.Tasks;
    using System.Transactions;
    using Genya.BASInfrastructure.Server.Business.WorkFlowInstanceNS;
    using Genya.BASInfrastructure.Shared.Model.CDM.WorkFlowsDomain;
    using Genya.Common.Log;
    using System.Globalization;
    using System.Collections.ObjectModel;
    using Genya.Workflow.Genya.Activities;
    using System.Runtime.Remoting.Messaging;
    using Genya.WorkFlow.Infrastructure;
    using Genya.Workflow.Genya.Infrastructure;
    using Genya.BASInfrastructure.Shared.Model.PDM.WorkFlow;
    using CSP.Framework.Shared.Model;

    namespace Genya.WorkFlow
    {
      public class WorkflowEngine : IDisposable {
        private bool m_WFCompleted = false;
        private WorkFlowInstanceFacade m_WorkFlowInstanceFacade = new WorkFlowInstanceFacade();
        AutoResetEvent instanceIsIdle = new AutoResetEvent(false);
        private WorkflowApplication m_WorkflowApplication;
        private WorkflowInstance m_WorkflowInstance;

        public WorkflowInstance WorkflowInstance { get { return m_WorkflowInstance; } }
        internal System.Transactions.Transaction m_Transaction;
        private Activity m_WorkflowActivity;
        internal WorkflowEngine(WorkflowInstance pWorkflowInstance)
          : this(pWorkflowInstance, null) { }
        internal WorkflowEngine(WorkflowInstance pWorkflowInstance
            , WorkflowIdentifiers pWorkFlowInputArguments) {
          using (var tc = new GenyaTraceContext()) {
            try {
              if (pWorkflowInstance == null) throw new Exception("pWorkflowInstance==null");
              m_Transaction = System.Transactions.Transaction.Current;
              m_WorkflowInstance = pWorkflowInstance;
              m_WorkflowActivity = Genya.BASInfrastructure.Shared.Model.PDM.WorkFlow.WFDefinitionManager.LoadWorkFlowActivityFromXaml(m_WorkflowInstance.VersionedWorkFlowName);

              //var cc = (DynamicActivity)m_WorkflowActivity;
              //Console.WriteLine(cc.Name);

              InstanceStore lInstanceStore = SetupInstanceStore();
              if (pWorkflowInstance.ChangeTracker.State == CSP.Framework.Shared.Model.ObjectState.Added) {
                var l_inputs = LoadInputs(pWorkFlowInputArguments);
                m_WorkflowApplication = new WorkflowApplication(m_WorkflowActivity, l_inputs);
              }
              else {
                m_WorkflowApplication = new WorkflowApplication(m_WorkflowActivity);
              }

              GenyaExtension lGenyaExtension = new GenyaExtension(CSPContext.CreateFromCSPContext());
              m_WorkflowApplication.Extensions.Add(lGenyaExtension);

              m_WorkflowApplication.Extensions.Add(pfCreateTrackingPartecipant(lGenyaExtension));

              m_WorkflowApplication.InstanceStore = lInstanceStore;

              m_WorkflowApplication.OnUnhandledException = (e) => {
                using (var tc2 = new GenyaTraceContext()) {
                  tc2.TraceMessage("UnhandledException=" + e.UnhandledException.ToString()
                  + " WFInstanceId=" + e.InstanceId
                  + " Activity.DisplayName=" + e.ExceptionSource.DisplayName
                  + " ExceptionSourceInstanceId=" + e.ExceptionSourceInstanceId);
                }
                return UnhandledExceptionAction.Abort;
              };


              m_WorkflowApplication.PersistableIdle = (e) => {
                using (var tc2 = new GenyaTraceContext()) {
                  instanceIsIdle.Set();
                  //return PersistableIdleAction.Persist;
                  return PersistableIdleAction.None; // PersistableIdleAction.Persist;
                }
              };


              m_WorkflowApplication.Completed = (e) => {
                using (var tc2 = new GenyaTraceContext()) {
                  tc2.TraceMessage("CompletionState=" + e.CompletionState);
                  if (e.TerminationException != null)
                    tc2.TraceError(e.TerminationException);
                  foreach (var par in e.Outputs) {
                    tc.TraceMessage("Key=" + par.Key + " Value=" + par.Value);
                  }
                  m_WFCompleted = true;
                  // TODO mandare un custom tracking record per la modalità con cui si è chiuso il workflow
                  instanceIsIdle.Set();
                }
              };

              m_WorkflowApplication.Aborted = (e) => {
                using (var tc1 = new GenyaTraceContext()) {
                  tc1.TraceError(e.Reason, TraceEventType.Error);
                }
              };


              m_WorkflowApplication.Unloaded = (e) => {
                using (var tc1 = new GenyaTraceContext()) {
                  instanceIsIdle.Set();
                }
              };


              if (pWorkflowInstance.ChangeTracker.State == CSP.Framework.Shared.Model.ObjectState.Added) {
                using (var ts = new System.Transactions.TransactionScope(TransactionScopeOption.Suppress)) {
                  m_WorkflowApplication.Persist();
                  ts.Complete();
                }
                pWorkflowInstance.IdWorkFlowInstance = m_WorkflowApplication.Id;
                // salve i WF identifiers sul db .. nella wfinstance non sono tirati su
                // servono solo epr fare le query successivamente

                var lWorkFlowIdentifierMap = WorkFlowMetadata.GetWorkFlowIdentifierMap(WFDefinitionManager.GetWFMetaDataPath(m_WorkflowInstance.VersionedWorkFlowName));


                //pWorkflowInstance.WorkflowInstanceIdentifiers = new TrackableCollection<WorkflowInstanceIdentifier>();
                foreach (var x in pWorkFlowInputArguments.ToEFIdentifiers()) {
                  string lcolname = "";
                  if (!lWorkFlowIdentifierMap.TryGetValue(x.IdentifierName, out lcolname)) {
                    throw new Exception("workflowidentifier name " + x.IdentifierName + "not found in metadata file (/workflow/workflowidentifiers/identifier))");
                  }
                  else {
                    switch (lcolname) {
                      case "WorkflowArg1":
                        m_WorkflowInstance.WorkflowArg1 = x.IdentifierValue;
                        break;
                      case "WorkflowArg2":
                        m_WorkflowInstance.WorkflowArg2 = x.IdentifierValue;
                        break;
                      case "WorkflowArg3":
                        m_WorkflowInstance.WorkflowArg3 = x.IdentifierValue;
                        break;
                      case "WorkflowArg4":
                        m_WorkflowInstance.WorkflowArg4 = x.IdentifierValue;
                        break;
                      case "WorkflowArg5":
                        m_WorkflowInstance.WorkflowArg5 = x.IdentifierValue;
                        break;
                      case "WorkflowArg6":
                        m_WorkflowInstance.WorkflowArg6 = x.IdentifierValue;
                        break;
                      case "WorkflowArg7":
                        m_WorkflowInstance.WorkflowArg7 = x.IdentifierValue;
                        break;
                      case "WorkflowArg8":
                        m_WorkflowInstance.WorkflowArg8 = x.IdentifierValue;
                        break;
                      case "WorkflowArg9":
                        m_WorkflowInstance.WorkflowArg9 = x.IdentifierValue;
                        break;
                      case "WorkflowArg10":
                        m_WorkflowInstance.WorkflowArg10 = x.IdentifierValue;
                        break;
                      default:
                        throw new Exception("No mapping for colname " + lcolname);
                    }
                  }
                }
                m_WorkFlowInstanceFacade.Save(pWorkflowInstance);

                m_WorkflowApplication.Run();
                instanceIsIdle.WaitOne();
              }
              else {
                using (var ts = new System.Transactions.TransactionScope(TransactionScopeOption.Suppress)) {
                  m_WorkflowApplication.Load(pWorkflowInstance.IdWorkFlowInstance);
                }
                m_WorkflowApplication.Run();
                instanceIsIdle.WaitOne();
              }
              tc.TraceMessage("Exiting");
            }
            catch (Exception ex) {
              tc.TraceError(ex);
              throw;
            }
          }
        }

        //public ReadOnlyCollection<System.Activities.Hosting.BookmarkInfo> GetBookMarks()
        // {
        //    return m_WorkflowApplication.GetBookmarks();
        //}

        public void ResumeActivity(string pBookmark, GenyaBookMarkCallBackContextBase pGenyaBookMarkCallBackContext) {
          Bookmark lbk = new Bookmark(pBookmark);
          BookmarkResumptionResult lBookmarkResumptionResult
              = m_WorkflowApplication.ResumeBookmark(lbk, pGenyaBookMarkCallBackContext);
          if (lBookmarkResumptionResult == BookmarkResumptionResult.NotFound)
            throw new Exception("The bookmark " + pBookmark + " was not found");
          instanceIsIdle.WaitOne();
        }

        private GenyaTrackingPartecipant pfCreateTrackingPartecipant(GenyaExtension pGenyaExtension) {
          const String all = "*";

          GenyaTrackingPartecipant l_GenyaTrackingPartecipant = new GenyaTrackingPartecipant(pGenyaExtension, m_WorkflowInstance) {

            // Create a tracking profile to subscribe for tracking records
            // In this sample the profile subscribes for CustomTrackingRecords,
            // workflow instance records and activity state records
            TrackingProfile = new TrackingProfile() {
              Name = "GenyaTrackingProfile",
              Queries =
                        {
                            new CustomTrackingQuery()
                            {
                             Name = all,
                             ActivityName = all,
                             QueryAnnotations = {
                                    { all , all }
                             }
                            },
                            new WorkflowInstanceQuery()
                            {
                                // Limit workflow instance tracking records for started and completed workflow states
                                States = { WorkflowInstanceStates.Started, WorkflowInstanceStates.Completed },
                                QueryAnnotations = {
                                    { all , all }
                                },
                            },
                            new ActivityStateQuery()
                            {
                                // Subscribe for track records from all activities for all states
                                ActivityName = all,
                                States = { all },

                                // Extract workflow variables and arguments as a part of the activity tracking record
                                // VariableName = "*" allows for extraction of all variables in the scope
                                // of the activity
                                Variables =
                                {                               
                                    { all }  
                                },
                                Arguments =
                                {
                                    { all }
                                },
                                QueryAnnotations = {
                                    { all , all }
                                }
                            }  
                        }
            }
          };
          l_GenyaTrackingPartecipant.ParentTransaction = m_Transaction;
          return l_GenyaTrackingPartecipant;
        }

        private Dictionary<string, object> LoadInputs(WorkflowIdentifiers pWorkFlowInputArguments) {
          using (var tc = new GenyaTraceContext()) {
            Dictionary<string, object> l_inputs = new Dictionary<string, object>();
            if (pWorkFlowInputArguments == null) return l_inputs;
            foreach (var x in pWorkFlowInputArguments) {
              tc.TraceMessage("Name=" + x.Key + " value=" + x.Value);
              l_inputs.Add(x.Key, x.Value);
            }
            return l_inputs;
          }
        }

        private string m_cnstring = "";

        private InstanceStore SetupInstanceStore() {
          using (var tc = new GenyaTraceContext()) {
            using (var ts = new System.Transactions.TransactionScope(TransactionScopeOption.Suppress)) {
              var lStateMachinesFacade = new WorkFlowInstanceFacade();
              m_cnstring = lStateMachinesFacade.WfStoreConnectionString();
              tc.TraceMessage("wf store cnstring=" + m_cnstring);
              m_instanceStore = new SqlWorkflowInstanceStore(m_cnstring);
              m_instanceStore.InstanceCompletionAction = InstanceCompletionAction.DeleteAll;
              m_handle = m_instanceStore.CreateInstanceHandle();
              InstanceView view = m_instanceStore.Execute(m_handle, new CreateWorkflowOwnerCommand(), TimeSpan.FromSeconds(30));
              m_instanceStore.DefaultInstanceOwner = view.InstanceOwner;
              ts.Complete();
              return m_instanceStore;
            }
          }
        }
        SqlWorkflowInstanceStore m_instanceStore = null;
        InstanceHandle m_handle = null;
        public void Dispose() {
          using (var tc = new GenyaTraceContext()) {
            using (var ts = new System.Transactions.TransactionScope(TransactionScopeOption.Suppress)) {
              if (m_WorkflowApplication != null) {
                if (m_WFCompleted != true) {
                  m_WorkflowApplication.Persist();
                }
                m_WorkflowApplication.Unload();
              }
              if (m_instanceStore != null) {

                m_instanceStore.Execute(m_handle, new DeleteWorkflowOwnerCommand(), TimeSpan.FromSeconds(30));
                m_handle.Free();
              }
              ts.Complete();
            }
          }
        }
      }
    }

    Thursday, August 28, 2014 9:24 AM
  • Do you happen to have any classes derived from PersistenceParticipant or PersistenceIOParticipant that are added to the WorkflowApplication.Extensions collection?

    The "Unloaded" tracking record is generated at the beginning of the unload processing. It is followed by the work of unloading the instance, which includes invoking the PersistenceParticipant and PersistenceIOParticipants instances that are registered.

    When the persist/unload is completed, the WorkflowApplication.Unloaded event is raised.

    I am wondering if there is some sort of deadlock situation between PersistenceParticipants when running multi-threaded.

    Is there any data shared in the host that needs to be synchronized between the different threads running workflow instances? If yes, what is the locking scheme for that synchronization?

    Do you utilize the Promoted Properties feature of SqlWorkflowInstanceStore?

    Thursday, August 28, 2014 6:27 PM
  • The best way for me to tackle this is to look at other workflow examples that uses multiple threads and bookmarks.

    Do you know of such example, if so could you provide the link?

    Friday, August 29, 2014 7:40 AM
  • Thank you for the link, I've already looked at that and could not find an example that is using multiple threads and bookmarks.
    Friday, August 29, 2014 3:32 PM
  • I've resolved the Persistence issue but I am still getting the WorkflowApplication.Load hanging when I process more than 18 instances on 4 multiple threads.
    Monday, September 1, 2014 7:49 AM
  • Unfortunately I am on vacation for a couple of weeks so I won't be able to explore this further until I get back.

    But would you mind sharing what you discovered to get your "unload" problem solved? I like to keep track of the pain points that users have with workflows so that we can take past experiences into account when other users have questions.

    Thanks.

    Wednesday, September 3, 2014 4:54 PM
  • The unload was due to a transaction open when trying to resume a bookmark. This was the reason why the persistable event was stuck waiting for the transaction to complete, which is why unload was not being called. Once I fixed the open transaction issue it started to work as expected.

    Now when running over 18 instances of the same workflow using 4 threads, when I try to load the workflow application by calling WorkflowApplication.Load it hangs in that method. I'm 100% sure there are no open transaction because I ran a SQL profiler and saw no blocks.

    As a workaround I'm running my program as a single thread. 
    • Edited by Din123 Thursday, September 4, 2014 7:10 AM
    Thursday, September 4, 2014 7:08 AM
  • as i said . I had the load fail (not block, fail immediately) until i wrapped the load method in a transaction.suppress .. give it a try

    IMHO, load and save wf peristence is completely broken when there are transactions around

    Thursday, September 4, 2014 2:56 PM
  • I've tried that and no joy. Also how many threads were you running?

    I'm running 4 threads processing the same workflow with separate workflow instances.

    • Edited by Din123 Monday, September 8, 2014 7:02 AM
    Monday, September 8, 2014 7:02 AM
  • I am using the thread pool to process my jobs. Does the WorkflowApplication.Load also use a thread pool underneath?
    Wednesday, October 1, 2014 10:30 AM
  • I am sorry I didn't catch on to the Transaction issue before. Din123 made no mention of using transactions in any posts. Only Sabbadin mentioned transactions, so it wasn't clear that Din123 was using transactions.

    Now to the thread pool question.

    When using WorkflowApplication, the workflow runtime defines an internal SynchronizationContext implementation that utilizes the I/O related threads in the ThreadPool thru the use of ThreadPool.UnsafeQueueNativeOverlapped for each of its internal "work items" that it needs to schedule asynchronously.

    When given a thread, the workflow runtime will execute a workflow instance on that thread until that instance becomes idle.

    Can you confirm via SQL Server tracing that the stored procedure used to load a workflow instance (LoadInstance) is successfully executing?

    Wednesday, October 1, 2014 11:41 PM
  • As i already said , what happends to me is that Loadinstance fails if there is an ambient transaction active .. that is if , i don't wrap it into a transactionscope suppres . it fails immediately , not after a timeout, giving an error saying that "i didn't provide the rquired parameters to the workflow", as if it was trying to create a new instance of the wf, not loading an existing one.
    Thursday, October 2, 2014 7:42 AM
  • Sabbadin,

    It sounds like you are running into a different issue. I have seen a situation where the Load attempt failed, but the exception generated by the Load was "masked" by the workflow runtime and if the workflow definition had arguments defined, it complained that the arguments weren't provided. The situation where I have seen this did not involve transactions and the user was using their own implementation of an InstanceStore, rather than using SqlWorkflowInstanceStore.

    I am interested in pursuing your situation a bit more. Would you mind starting up a separate forum thread, explaining the symptoms you experienced? Please provide details on the InstanceStore you are using and what other operations have been done inside the ambient transaction that you have outstanding when you attempt the call to WorkflowApplication.Load. Please include information about the possibility that the transaction was promoted to become a distributed (MSDTC) transaction, rather than just a local System.Transactions transaction.

    Thank You.

    Friday, October 3, 2014 1:14 AM
  • The root of the problem was due to me setting the ThreadPool.SetMaxSize. I was using up all the threads which resulted in the WorkflowApplication failing to find a thread to use when the Async call came back from the SQL db. I've now changed my code to only use a certain number of threads in the ThreadPool and let the WorkflowApplication use the rest and the problem is now resolved.
    Friday, October 3, 2014 10:20 AM
  • Din123,

    Thank you for letting us know how you resolved your issue.

    Jim

    Friday, October 3, 2014 5:31 PM