none
How To Wait for Tasks to Complete? RRS feed

  • Question

  • In the sample below, the number of lines that end up written on the console depends almost entirely on the timing on your machine.  The reason is that the main thread exits after a hard-coded Thread.Sleep() without explicitly waiting for the queued tasks to complete their processing.

     

    Code Snippet

    static void Main()

    {

      using( var dq = new DispatcherQueue( "TEST Dispatcher Queue" ) )

      {

        var port = new Port< int >();

        var rand = new Random();

        Arbiter.Activate( dq,

          Arbiter.Receive ( true, port, item =>

            {

              Thread.Sleep( rand.Next( 500 ) );

              Console.WriteLine( "i:" + item );

            }

          )

        );

        for( int i = 1; i <= 200; ++i )

          port.Post( i );

      }

     

      // hack: wait for CCR tasks to complete

      Thread.Sleep( 1000 );

      //Console.ReadLine();

    }

     

    So, how does one wait for all tasks to complete, in a generic sense?  Obviously, we could complicate this further by adding a new message type to the end of the queue that would then respond by setting a wait event, or something of that nature.  However, circumstances are not guaranteed that the terminal message posted would be in fact the last message (e.g., in cases that a work item queues up additional work items).  Further, there is no guarantee of ordered processing of posted messages, that I am aware of yet.

     

    It seems there should be a built-in method available on a DispatcherQueue to determine if all requests have been satisfied.  Obviously there could be things queued up in a Port or by a slowly executing Arbiter, but it seems either respective methods could be added where appropriate or validation code added once the wait method returns sucessfully.

     

    I know CCR is all about removing Wait functions, but this is one of those special cases where a Wait function is required and (I believe) should be supplied by the CCR itself.

     

    -Jonathan

    Wednesday, November 19, 2008 1:23 AM

Answers

  • Here are two simple solutions to make an asynchronous CCR task callable as a synchronous method. Both assume that you have a concept of being done. As George already said, use a completion port to signal when you are done with a sub task. The initiating task can then signal its own completion when all subtasks are done (and so forth ...)

    1) This addresses the issue that a console application exits before your CCR tasks are done. In the original post that happens because you use the default dispatcher which uses threadpool threads. These threads are background threads and thus exit when the main thread exits. You can avoid this by using an explicit dispatcher.

        var dispatcher = new Dispatcher();
        var queue = new DispatcherQueue("queue", dispatcher);

    Now the main thread will not exit. In order to exit the application call

        dispatcher.Dispose();

    when you are done. This is a quick and dirty approach (i.e. do not use it!). I recommend using the following approach.

    2) This is useful when integrating CCR into existing synchronous application frameworks. When a synchronous method is called, it spawns a CCR task. The method can only return when the CCR task completes. We assume that the CCR task signals its completion one a port.

    public void SyncTask()
    {
        
    var done = new ManualResetEvent(false);
        
    Exception e = null;

        
    Arbiter.Activate(queue,
            
    Arbiter.Choice(
                AsyncTask(), // start the async task
                success => done.Set(),
                exception =>
                {
                    e = exception;
                    done.Set();
                }
            )
        );

        done.WaitOne(); // this blocks! the calling thread until the ccr task is complete

        
    if (e != null)
        {
            
    throw e;
        }
    }

    private SuccessFailurePort AsyncTask()
    {
        
    var resultPort = new SuccessFailurePort();

        
    // this is async (e.g. spawn iterator, send request etc)
            
    // when complete send message
            
    resultPort.Post(SuccessResult.Instance);
        
    // end of async
        
        return resultPort;
    }

    If you need this more than once you can wrap the implementation of SyncTask into a generic method quite easily.

    Approach 2) is what DssHost and many other tools shipped with Robotics Dev. Studio do internally when calling from the main thread into CCR.

    I hope this clarifies it a bit more.

    Andreas

    • Edited by Andreas Ulbrich Tuesday, January 20, 2009 12:54 AM fixed instruction order in error handler
    • Marked as answer by Trevor Taylor Monday, January 26, 2009 7:46 AM
    Tuesday, January 20, 2009 12:48 AM

All replies

  • A couple of things:

     

    CCR has a Timeout that can be used instead of Thread.Sleep, but it needs to be inside an Iterator. For example:

     

    Code Snippet
        yield return Timeout(1000);

     

    (This syntax is a new feature of RDS 2008 which simplifies the previous syntax for using Timeout.)

     

    You can Spawn an iterator. Running code directly inside main() is not the best way to go.

     

    Using Thread.Sleep inside a Receiver as you have done in your code snippet will tie up threads. Consequently it is possible to end up running out of threads and everything will hang until a Sleep completes.

     

    If you want to know when all tasks have completed, create a Completion Port. (It does not matter what the data type is of the port because you will not use the messages). Each task can send a message to the port. Then all you have to do is wait for multiple items on the port. If you know exactly how many tasks will be executed, this can be done using a Multiple Item Receiver with an explicit number of items.

     

    Ports do have a count that can be used to see if there are messages waiting, but you don't want to be spinning on this as a "busy wait".

     

    I think if you dig around inside the DIspatcherQueue using Intellisense (or use the Object Browser), there is a counter somewhere for the number of pending tasks. I just can't remember off the top of my head.

     

    And just as an aside, we do not recommend the use of Console.WriteLine, although I do it quite often for debugging. You can use the Log functions for this which helps because you can then use filtering on the message severity, e.g. LogInfo, LogError, etc.

     

    Trevor

     

     

    Wednesday, November 19, 2008 3:19 AM
  • Trevor,

     

    That was simply an example that's supposed to represent much more complicated scenarios.  I appreciate your response, but completion ports and other similar hacks are non-generic and hard to implement in a large scale app with a bunch of moving parts and different levels and modules that aren't fully aware of eachother.

     

    Look, I simply need a reliable way to ensure that the Dispatcher and DispatcherQueue is disposed at the right time.  This is, IMHO, a gaping hole in the current CCR design.  While "real" apps can't have ReadLines at the end of them, the completion port idea gets complicated very quickly when multiple and independently written modules are involved.

     

    I'm sure lots of people are using CCR is terribly intense ways that make CCR shine.  But, the scenarios are limited without the ability to ensure that queued tasks, in the general sense, have been fully serviced before the runtime is disposed.

     

    Let's say, for example, you are writing an asynchronous TraceListener for ASP.NET, console and service type applications.  Let's say the trace information is very important and you want to ensure it is written before the executing process closes.  There does not appear to be a proper hook to inform the listener that it is time to shutdown, so there isn't even a way to ensure that queued items are handled before termination.  Further, since the Dispatcher Queue must run as foreground threads, since the job it handles is so important, the DQ thread causes the main application to simply hang waiting for some signal to close all foreground threads.  There's probably a way to intercept a ThreadAbortException within the background thread, but (I think) that's not exposed in a way that can be handled by anyone other than the CCR.

     

    Maybe these really are just application-level issues that are the sole responsibility and pervue of an application developer, and maybe I'm just having trouble seeing that.  But, it seems pretty dang integral to the plumbing of a "coordination" runtime to offer a way to at least coordinate the shutdown process in a built-in, first-class way.

     

    Can anyone else chime in here and clarify this issue for me (i.e., tell me I'm totally wrong); or offer some generic solutions; or pitch in making the case that there needs to be a good, first-class citizen sort of way to know for sure that all queued requests have been satisfied.

     

    Thanks,

    Jonathan

    Thursday, November 20, 2008 3:07 AM
  • I had a similar quandry as Jonathan when I first employed the CCR in a service that handled incoming requests and then dispatched them to a CCR PortSet.  The code in the registered receivers actually could dispatch messages to the same ports for further processing.  The problem was handling the a request for the service to "stop" processing requests.

     

    In my case, I decided "stop" would mean that after the "stop" request was received, additional incoming requests to the service returned the equivalent of a "server not available" message to the sender, yet, requests that had already been dispatched to a CCR port could continue processing, even posting again to the port as necessary to complete the tasks.

     

    So, my first difficulty was in determining, "how do I know all these tasks are done?".    In my code, I just kept track of the number of "outstanding requests" (requests not yet completed) and waited for this to reach zero.  This is fine, but application-specific.

     

    I think what I had hoped for (and I believe what Jonathan was aiming for) was a generalized solution for a smart, coordinated execution of concurrent tasks in which a centralized "CCR manager", so to speak, handles, via some accounting mechanism, the "I'm done" semantics.  The general idea, at a high level, is this in this obviously not-real code, in some event loop thread.  We assume there's another thread that can signal some event to tell us to quit:

    Code Snippet

    SmartCcrManagerClass.RegisterMeAsCcrPosterFor( typeof(MsgType),

      delegateThatTakes_MsgType );

    do

    {

      // block until a request comes in or external event

      // defined elsewhere tells us we need to quit app or service

      MsgType someRequest = WaitForIncomingRequestToMyAppOrService();

     

      if( someRequest == null )

      {

        break; // this means we've been asked to 'quit' app or service

      }

     

      // eventually calls delegateThatTakes_MsgType

      // additionally, we can assume this handler also

      // makes additional calls to SmartCcrManagerClass.DispatchMsg()

      // with possibily the same or even different types

      SmartCcrManagerClass.DispatchMsg( someRequest );

     

    } while( true ); // we'll pretend this is elegant even though it's not

     

    // Here's where we wanted the 'magic'... it knows about 'outstanding' tasks,

    // so it blocks until those have been completed

    SmartCcrManagerClass.WaitForExistingTasksToFinish();

     

     

    Thursday, November 20, 2008 11:40 PM
  • Hi, you can achieve all this, but as one other poster mentioned, it is application specific. But that does not mean its hard:

    1) for every task you spawn (automate this by implenting your custom ITask), post a message on a port. Implement the ArbiterCleanuphandler on the Task, which is acalled when the task is complete. In there, remove one item. If items fall to zero, you are done, globally

     

    2) use a custom causality! When your app starts create a causality (implement your custom CausalityContext, with a finalizer). All other tasks from that point on are part of this causality. When causality evaporates, GC will kick in. When it does, your finalizer gets called, which means its time to shut down.

     

    so either the "flight count" approach with a 10 line custom Task implementation, or the causality one (much mor erobust since it will wait until pending receives are satsifed as well) will work.

     

    hope this helps.

     

    I have thought about the issue you mention but knowing when you are done is actually much harder than even the scenarios you allude to. There are scenarios where the system is completely "asleep" but due to registered receivers, or time outs waiting to fire or some other "contract" you are not done and you cant tell you ar enot done since active Task execution is only a small part of determining all that.

     

    thanx

    g

     

     

    Saturday, November 22, 2008 4:18 AM
  • George,

     

    Issue #1 -- Application-Level Gatekeeper

    Putting the more complicated TraceListener scenario aside, the fact that there is a solution based on the use of a "custom casualty" would seem to prove that there is a theoretical way to include a built-in event to signal completion status, at least within a call-context boundary concept.

     

    In the interim, and before I saw your custom casualty option, I created a set of "CcrCoordinator" classes that provided a bit of the the gatekeeping that Tim described in his response to this thread.  However, I was wrapping so much of the access to CCR classes that I lost some of the flexibility of the CCR.  Further, I was simply shimming the handlers just so I could track the completion status in terms of a CounterEvent class.  The counter class is only a number and ManualResetEvent that is set when counter is zero and not set when it's more than zero.

     

    To me, this is something that should be baked into the CCR and not something anyone outside of the CCR should have to assume the responsibility of getting right.  As you describe George, there's a lot to consider in some specific applications.  But, these exceptions are the only thing that should be application-specific.  For the general case, having a built-in CcrCoordinator type class with a WaitForTasksToComplete() method is essential to simplifying common usages, as well as filling the primary staple even for the exceptional application issues.

     

    Issue #2 -- General, Module-Friendly Shutdown Mechanism

    George, as the CCR gets more common, there will be modules that people will want to upgrade in a plug-and-play manner.  While the more complicated issues here may extend beyond your control, it seems that you will need to lobby internally to provide baked-in mechanisms within the CLR for modules to hide their internals while still being in full communication with the framework and application process regarding shutdown sequences (and well before the finalizer gets called).  It may be easy to map existing async code to CCR, but we also need a way to adapt synchronous interfaces to async internals where ensuring async task complete is critical to successful implementation.

     

    What is the future of the CCR (and CLR) with regard to these issues?

     

    -Jonathan

    Monday, November 24, 2008 5:27 PM
  • Hi Trevor,
    You wrote:
    If you want to know when all tasks have completed, create a Completion Port. (It does not matter what the data type is of the port because you will not use the messages). Each task can send a message to the port. Then all you have to do is wait for multiple items on the port. If you know exactly how many tasks will be executed, this can be done using a Multiple Item Receiver with an explicit number of items.
    Ports do have a count that can be used to see if there are messages waiting, but you don't want to be spinning on this as a "busy wait".

    The issue with this approach is implementing a "wait" logic. Currently, I did not find anything within CCR library that allows such functionality. Implementing Thread.Sleep or Thread.SpinWait does not work efficient enough, and having a loop in place adds up huge overhead on CPU.
    Thursday, January 8, 2009 4:30 PM
  • This is where you need to use an Iterator and the yield on the Completion Port. Alternatively you can set up (and activate) a Receiver on the port and then post a Drop message to yourself. The second approach does not require you to use an Iterator.

    Trevor
    Wednesday, January 14, 2009 3:09 AM
  • Here are two simple solutions to make an asynchronous CCR task callable as a synchronous method. Both assume that you have a concept of being done. As George already said, use a completion port to signal when you are done with a sub task. The initiating task can then signal its own completion when all subtasks are done (and so forth ...)

    1) This addresses the issue that a console application exits before your CCR tasks are done. In the original post that happens because you use the default dispatcher which uses threadpool threads. These threads are background threads and thus exit when the main thread exits. You can avoid this by using an explicit dispatcher.

        var dispatcher = new Dispatcher();
        var queue = new DispatcherQueue("queue", dispatcher);

    Now the main thread will not exit. In order to exit the application call

        dispatcher.Dispose();

    when you are done. This is a quick and dirty approach (i.e. do not use it!). I recommend using the following approach.

    2) This is useful when integrating CCR into existing synchronous application frameworks. When a synchronous method is called, it spawns a CCR task. The method can only return when the CCR task completes. We assume that the CCR task signals its completion one a port.

    public void SyncTask()
    {
        
    var done = new ManualResetEvent(false);
        
    Exception e = null;

        
    Arbiter.Activate(queue,
            
    Arbiter.Choice(
                AsyncTask(), // start the async task
                success => done.Set(),
                exception =>
                {
                    e = exception;
                    done.Set();
                }
            )
        );

        done.WaitOne(); // this blocks! the calling thread until the ccr task is complete

        
    if (e != null)
        {
            
    throw e;
        }
    }

    private SuccessFailurePort AsyncTask()
    {
        
    var resultPort = new SuccessFailurePort();

        
    // this is async (e.g. spawn iterator, send request etc)
            
    // when complete send message
            
    resultPort.Post(SuccessResult.Instance);
        
    // end of async
        
        return resultPort;
    }

    If you need this more than once you can wrap the implementation of SyncTask into a generic method quite easily.

    Approach 2) is what DssHost and many other tools shipped with Robotics Dev. Studio do internally when calling from the main thread into CCR.

    I hope this clarifies it a bit more.

    Andreas

    • Edited by Andreas Ulbrich Tuesday, January 20, 2009 12:54 AM fixed instruction order in error handler
    • Marked as answer by Trevor Taylor Monday, January 26, 2009 7:46 AM
    Tuesday, January 20, 2009 12:48 AM
  • This approach assumes there is only one task in the dispatcher, sort of like a asynchronous call. How could this be applied if the dispacher has a long queue (100 or so tasks) and the concurrency of said tasks would imply they do not necessarily complete in the order they were added?
    Tuesday, March 10, 2009 2:59 PM
  • Can you tell when all 100 tasks have completed? If not, then this is a never-ending story ... :-)

    Trevor
    Saturday, March 14, 2009 5:39 AM
  • Trevor Taylor said:

    Can you tell when all 100 tasks have completed? If not, then this is a never-ending story ... :-)

    Trevor



    My experience with the CCR thus far is still in its early stages, but can't you use dispatcher.ProcessedTaskCount to know how many tasks have completed?

    Best,
    Sebastian G.
    Saturday, March 14, 2009 6:59 PM
  • That's true. The point I was trying to make is that if you know that a compound operation is complete after xx tasks have been processed, then you can signal this by posting to another port. If you the tasks just keep coming, then there is no logical end.

    Trevor
    Sunday, March 15, 2009 3:14 AM
  • In my particular case, the tasks are all similar, so what is really needed is a way to know the dispatchqueue is empty, like an event or callback to be activated when this happens. So far I've found nothing of the sort in the documentation. The ProcessedTaskCount and other properties do not return useful values to know this.
    Friday, March 20, 2009 5:16 PM
  • I asked a very similar question on another thread. There doesn't seem to be any supported way to determine if a Dispatcher's queue is currently empty. If anyone has a solution that doesn't involve reflecting into a private member, I'd love to hear it.

    Thanks,
    Erick
    Wednesday, July 15, 2009 12:46 AM
  • I am not sure if the CCR team can give you anything that is really different than the Dispatcher.PendingTaskCount

    Anything else is really just too application specific.

    Besides with the power of ports and the coordination ability provided by the arbiters - solving this problem is fairly trivial.

    it is as simple as counting.  so, stop being lazy!
    Learning one small step at a time
    Thursday, July 16, 2009 1:01 AM