locked
Console apps

    Question

  • As I understand it, when I use "await" from the UI thread, and the framework is later ready to resume my method, it resumes on the UI thread -- even though the I/O operation I'm awaiting may have been running on the I/O completion thread. This makes a lot of sense from an API usability standpoint. I'm a little less clear on the mechanism, but I'm assuming that it snags SynchronizationContext.Current when I await, and later uses it to post a message to the UI message queue, which then causes the continuation to get called. Assuming I've got that much right, I see how it would work in WinForms (Windows message queue) and in WPF and Silverlight (Dispatcher).

    But how does that work in a console application? There's no message queue. Can I use "async" / "await" cooperative multitasking in a console app? WinForms and WPF let me run a fire-and-forget method and then fall back to the top-level message loop, and let it dispatch the continuation later; but there's nothing comparable in a console app. (Or is there some kind of ConsoleMessageLoop in the CTP?)

    I can easily imagine wanting to write a console app that downloads three different RSS feeds (simultaneously), aggregates them when they're all done downloading, outputs them to a file, and then exits. If I'm using the ...Async calls, then there's really no need for me to spin up new Task threads; cooperative multitasking would suffice. So it would be cool to use "async" and "await" to do it. But is that possible, without a message loop? How would this work?

    Sunday, October 31, 2010 12:48 AM

Answers

  • Hi Joe-

    re: "unless you write your own SynchronizationContext"

    You don't actually need to write your own SynchronizationContext to achieve this.  The System.Threading.Tasks.Dataflow.dll (included in the Async CTP) contains a ConcurrentExclusiveSchedulerPair, which exposes two properties: ExclusiveScheduler and ConcurrentScheduler.  The idea is that this is, in effect, an asynchronous reader/writer lock.  Any tasks that get scheduled to ExclusiveScheduler will only run when no other tasks on either scheduler are running, and any tasks that get scheduled to ConcurrentScheduler may run concurrently as long as there are no exclusive tasks running (and exclusive tasks take priority).  If all you care about is sequential processing, you can just use the ExclusiveScheduler. So, in a console app, you can write code like:

    private ConcurrentExclusiveSchedulerPair m_cesp = new ConcurrentExclusiveSchedulerPair();
    public async void Foo()
    {
      await m_cesp.ExclusiveScheduler.SwitchTo();
      ... // will run exclusively with regards to everything else on m_cesp
      await somethingElse;
      ... // will also run exclusively, as the above await will return to the current scheduler at the time of the await
    }
    

    In effect, this ExclusiveScheduler can serve as the SynchronizationContext you didn't want to have to write yourself.

    You can also schedule tasks directly onto the exclusive scheduler by using Task.Factory.StartNew and passing in the target scheduler.  Any async methods called within that task will see the exclusive scheduler as current and thus will automatically run all work on it.

    I hope that helps.

    Sunday, October 31, 2010 7:14 PM

All replies

  • You're exactly right: it snags SynchronizationContext.Current at the start of an await, and posts back to it afterwards.

    In a console application, SynchronizationContext.Current is null. This means that the stuff after an await just continues on whatever thread it happens to be on (rather than synchronizing back anywhere). So you can still use async/await, so long as you're ready for these continuations to be run on different threads.

    (Note: it's possible to install your own synchronization context, if you want to have better control over the process).

     

    Incidentally, if you wrote this code:

    static async void Main()
    {
      Console.WriteLine("a");
      await f();
      Console.WriteLine("b");
    }

    then it would print "a" and exit IMMEDIATELY (without awaiting). That's because "await" causes control to return to the caller immediately. In VB in the CTP it gives an error message if you try to make Main async. C# will do that as well. The correct way to do this is to write f().Wait() in your Main.

    (That's surprising. Normally it's a recipe for deadlock if you mix synchronous-blocking-stuff with async stuff. But in this case it's okay, since we know that the body of f() will be able to run on any thread, and isn't dependent on its caller thread being free).

     

    Sunday, October 31, 2010 1:14 AM
  • could somebody please explain me what 'snag' means ? in google translate i got : 'hit an obstacle'.

     

    Arek

    Sunday, October 31, 2010 7:51 AM
  • Please look at the MSDN page for SynchornizationContext.

    In short, previous to .NET 2 the calle (the one who execute the async method) had to use specific mechanism to return control to the caller, that is switch the thread context back. Environments like WinForms need code to execute on specific thread so they had methods like Control.Invoke to marshal a call back to the UI thread.

    When WPF came along it also had similar thread model, but not quite the same so SynchornizationContext was introduce which eliminate the need to know which specific thread model was used. The caller set the SynchornizationContext.Current and the calle simply use it.

     

    Ido

    Sunday, October 31, 2010 8:58 AM
  • @Lucian: Okay, that makes sense. There's no thread-sensitive UI framework to appease, so running the continuation on another thread should generally be okay, as far as that one async method is concerned (and as long as you don't use any thread-local state).

    Still, it does mean that cooperative multitasking -- where other async methods only run when you yield -- does not exist in a console app (unless you write your own SynchronizationContext). So you have to do locking and all the other concurrency control just like you do with multithreaded Tasks; the only thing console apps gain from the new framework is the "await" keyword, which saves you from having to manually write a lambda and enlist it as a Task's continuation (and unifies exception handling logic, etc.) "Await" should still be really nice, don't get me wrong, but you would still have the headaches of concurrency. Sound about right?

    @agend: Here "snag" means "grab hold of". It captures the current value of SynchronizationContext into a local variable or field or something, so it can refer back to that original value later. (The translation you found is too narrow. Really, a "snag" is something that hooks onto you as you pass by -- for example, catching, or "snagging", your jacket on a tree branch -- and it holds you back unexpectedly. From there, you can kind of see where "snag" can also mean both "grab", as used here, and "obstacle", as in the definition you found.)

    Sunday, October 31, 2010 2:52 PM
  • Hi Joe-

    re: "unless you write your own SynchronizationContext"

    You don't actually need to write your own SynchronizationContext to achieve this.  The System.Threading.Tasks.Dataflow.dll (included in the Async CTP) contains a ConcurrentExclusiveSchedulerPair, which exposes two properties: ExclusiveScheduler and ConcurrentScheduler.  The idea is that this is, in effect, an asynchronous reader/writer lock.  Any tasks that get scheduled to ExclusiveScheduler will only run when no other tasks on either scheduler are running, and any tasks that get scheduled to ConcurrentScheduler may run concurrently as long as there are no exclusive tasks running (and exclusive tasks take priority).  If all you care about is sequential processing, you can just use the ExclusiveScheduler. So, in a console app, you can write code like:

    private ConcurrentExclusiveSchedulerPair m_cesp = new ConcurrentExclusiveSchedulerPair();
    public async void Foo()
    {
      await m_cesp.ExclusiveScheduler.SwitchTo();
      ... // will run exclusively with regards to everything else on m_cesp
      await somethingElse;
      ... // will also run exclusively, as the above await will return to the current scheduler at the time of the await
    }
    

    In effect, this ExclusiveScheduler can serve as the SynchronizationContext you didn't want to have to write yourself.

    You can also schedule tasks directly onto the exclusive scheduler by using Task.Factory.StartNew and passing in the target scheduler.  Any async methods called within that task will see the exclusive scheduler as current and thus will automatically run all work on it.

    I hope that helps.

    Sunday, October 31, 2010 7:14 PM
  • @Lucian: Okay, that makes sense. There's no thread-sensitive UI framework to appease, so running the continuation on another thread should generally be okay, as far as that one async method is concerned (and as long as you don't use any thread-local state).

    Still, it does mean that cooperative multitasking -- where other async methods only run when you yield -- does not exist in a console app (unless you write your own SynchronizationContext). So you have to do locking and all the other concurrency control just like you do with multithreaded Tasks; the only thing console apps gain from the new framework is the "await" keyword, which saves you from having to manually write a lambda and enlist it as a Task's continuation (and unifies exception handling logic, etc.) "Await" should still be really nice, don't get me wrong, but you would still have the headaches of concurrency. Sound about right?

    That is mostly right.  If you're dealing with an API that's thread based, you'll still have all of the concurrency issues to worry about.  This is true when dealing with Windows Forms applications and the like as well - so the problem isn't unique to console applications.

     

    The async/await calls do not eliminate the need to worry about shared state or threading issues.  They do simplify some of the most common problems (synchronization with a UI), but there is still the potential for threading issues to arise.

     

    The one other thing I'll mention, though - await isn't just about threading.  A console application may very well be running async methods that aren't threading-based, in which case no new threads are coming into the picture, and threading issues aren't a problem here.  For example, see this thread which asks about using await with a Process instead of a thread...

     

     


    Reed Copsey, Jr. - http://reedcopsey.com
    If a post answers your question, please click "Mark As Answer" on that post and "Mark as Helpful".
    Sunday, October 31, 2010 8:10 PM
  • @Stephen: I'm not getting my brain wrapped around your example. Where's the event loop? In WinForms, there's a top-level event loop; your GUI event handlers can be fire-and-forget and it's fine, because they were called by the event loop and it'll be there to run the continuation. But all I see in your example is a fire-and-forget method. There's no event loop calling it. When that fire-and-forget method exits, it will return to Main(), not to an event loop, and nothing will ever call the continuation. What am I missing?

    @Reed: But in WinForms, the continuation always fires on the same thread that the async method started on. So all of the code written with "async" will run on the UI thread, unless I manually add some threading code *of my own*. I could be awaiting a background-thread task like FileStream.BeginRead, but *my* code -- my continuation -- will all still run on the UI thread.

    But with a console app, if I await something that's scheduled on a thread-pool thread, then my continuation runs on that thread-pool thread too. So as far as I can tell, some continuations will be running on the main thread, others on thread-pool threads, and now even *my* code is multithreaded, instead of being serialized on the UI thread like in WinForms or WPF. Now I can context-switch anywhere, or even run multiple lines of code concurrently. In WinForms or WPF, the only context switches are at "await" statements, which means I don't need to do as much work around synchronization. That's what I'm getting at.

    Or is there something inherent in the mechanism that means I *won't* have multiples of my methods running concurrently, even if they're switching off to other threads? My head hurts...
    Sunday, October 31, 2010 10:12 PM
  • Hi Joe-

    re: "@Stephen: I'm not getting my brain wrapped around your example"

    Maybe I was misunderstanding your question. I thought you were asking how you could force all of your async methods into a situation where only one was ever running at a time, such that multiple async methods about to resume from an await would be serialized, with only one able to run their await continuations at a time.  That's what my solution was getting at.  When you called FooAsync(), it would immediately jump to the exclusive scheduler, and all awaits in the method would be forced back to the exclusive scheduler.  This scheduler ensures that only one task queued to the scheduler gets to execute at a time.

    Sunday, October 31, 2010 10:20 PM
  • @Stephen: I see. I think I see. So I would still be using the null SynchronizationContext, meaning all of my awaits (and those from any other methods I call) would be scheduled willy-nilly on any old thread that the tasks I'm awaiting happen to be running on; but the scheduler would somehow do some locking to make sure only one of those continuations would run at a time, so I would still get the effect of only yielding at an await, and wouldn't need to protect every line of code from concurrency issues?

    If that's true, then it seems like a very useful thing to have, although I really don't understand how the control flow would work.

    Sunday, October 31, 2010 10:25 PM
  • Hi Joe-

    re: "would be scheduled willy-nilly on any old thread that the tasks I'm awaiting happen to be running on; but the scheduler would somehow do some locking"

    When you create a ConcurrentExclusiveSchedulerPair, you tell it where to run the underlying work.  This defaults to the thread pool using TaskScheduler.Default.  Then, when you schedule work to one of ConcurrentExclusiveSchedulerPair's schedulers, it in turn schedules tasks to the underlying scheduler.  These tasks that it schedules take the tasks that you queued and processes them, either one-at-a-time if they were queued to the ExclusiveScheduler, or en mass if they were queued to the ConcurrentScheduler.  It doesn't lock while executing work items; rather, it just queues up the tasks and processes them when it can, i.e. if there are five tasks queued to the exclusive scheduler, it'll pick off the first one and run it, and then when that ones done, it'll go back, grab the next exclusive task, process it, and so on.

    In async methods, think of each block of code between await points as being a task, e.g.

    public async void FooAsync()
    {
        await something1;
        ... // this is a task
        await something2;
        ... // this is another task
        await something3;
        ... // this is yet another task
        ... // etc.
    }

    When you hit an await, it first checks whether SynchronizationContext.Current is null; if it's not, when the awaited task completes, execution will resume by Post'ing to the captured context.  If SynchronizationContext is null, TaskScheduler.Current is captured (if you're not inside a task, this will just be TaskScheduler.Default), and when the awaited task completes, execution will resume by running a task on the captured scheduler.

    Sunday, October 31, 2010 10:55 PM
  • Hello, 

     

    please give some crticism to my code..

    My console application needs to start some main async Task (Run method) and have ability to manage it via. keyboard for example cancel it when any key is pressed.

    Can this be implemented better with some special class from .NET framework or async CTP?

     

    Thanks!

     

     

    class Processor

    {

      async Task Run(CancellationToken ct)

      {

         ...

      }

    }

     

     

    static void Main(string[] args)

    {

      var processor = new Processor();

      var cts = new CancellationTokenSource();

      var run = TaskEx.RunEx( async () => 

        {

     

          try

          {

            await processor.Run(cts.Token);

            Console.WriteLine("Work is done");

          }

          catch(TaskCanceledException)

          {

            Console.WriteLine("Task was stopped.");   

          }

        }

     

       );

      Console.ReadKey();

      cts.Cancel();

      run.Wait();

    }


     


    Sergey.
    Saturday, January 15, 2011 1:44 PM