locked
CallContext - What are the recommendations for using this going forwards?

    Question

  • Does anyone know if there's any planned support in upcoming versions of .NET for storing objects in the ambient execution context? There's a definite need for it, as highlighted in the following pages:

    Limitations of CallContext: https://connect.microsoft.com/VisualStudio/feedback/details/276325/provide-better-context-support-to-async-operations

    Implicit vs Explicit Context: http://msmvps.com/blogs/jon_skeet/archive/2010/11/08/the-importance-of-context-and-a-question-of-explicitness.aspx

    The types of object which could be added to ambient execution context are:

    • Those which are not suitable for DI. I.e. extrinsic dependencies which should be defined by the call-site, but cross-cut the application-domain in a way which makes them impractical to be passed as method arguments. Examples: CancellationToken, IProgress, Transactions, SynchronizationContext, SecurityContext. (In fact, ExecutionContext has been hardcoded to pass the latter 2 around... any good reason why an open collection cannot be exposed for custom code?)
    • Those which are used by generic code emitted into method bodies... code which has no knowledge about the explicit dependencies available to it as arguments on the housing method, or properties of its declaring type. Consider PostSharp Aspects in the AOP world.

    Would be great to hear people's thoughts on this one :)

    Thursday, January 19, 2012 8:19 AM

Answers

  • Stephen -

    Thanks for responding. I hope you don't mind a long post. :)

    Consider the SO question: the op is looking for a way to have an async-friendly "NDC" (Nested Diagnostic Context) - this is similar to CorrelationManager.LogicalOperationStack, but NDC is commonly used at a more granular level (so the example in this post is contrived but not unrealistic).

    Here's a simple MyNDCStack type with push/pop/print operations based on LogicalCallContext:

    public sealed class MyNDCStack
    {
        private const string name = "MyNDCStack";
        private readonly ConcurrentStack<string> stack;
    
        public MyNDCStack()
        {
            this.stack = new ConcurrentStack<string>();
        }
    
        public IDisposable PushContext(string context)
        {
            this.stack.Push(context);
            return new Pushed(this);
        }
    
        private void Pop()
        {
            string junk;
            this.stack.TryPop(out junk);
        }
    
        private static MyNDCStack CurrentContext
        {
            get
            {
                var ret = CallContext.LogicalGetData(name) as MyNDCStack;
                if (ret == null)
                {
                    ret = new MyNDCStack();
                    CallContext.LogicalSetData(name, ret);
                }
    
                return ret;
            }
        }
    
        public static IDisposable Push(string context)
        {
            return CurrentContext.PushContext(context);
        }
    
        public static string Current
        {
            get
            {
                return string.Join(" ", CurrentContext.stack.ToArray().Reverse());
            }
        }
    
        private sealed class Pushed : IDisposable
        {
            private MyNDCStack stack;
    
            public Pushed(MyNDCStack stack)
            {
                this.stack = stack;
            }
    
            public void Dispose()
            {
                if (this.stack != null)
                    this.stack.Pop();
                this.stack = null;
            }
        }
    }
    

    This works fine with serial/nested async operations, but it doesn't work so well when I use it with async operations running in parallel:

    class Program
    {
        static void Log(string message)
        {
            Console.WriteLine(MyNDCStack.Current + ": " + message);
        }
    
        static void Main(string[] args)
        {
            try
            {
                var tasks = new[]
                {
                    SomeWork("1"),
                    SomeWork("2"),
                };
    
                Task.WaitAll(tasks);
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex);
            }
            Console.ReadKey();
        }
    
        static async Task SomeWork(string stackName)
        {
            using (MyNDCStack.Push(stackName))
            {
                Log("<SomeWork>");
                await MoreWork("A");
                await MoreWork("B");
                Log("</SomeWork>");
            }
        }
    
        static async Task MoreWork(string stackName)
        {
            using (MyNDCStack.Push(stackName))
            {
                Log("<MoreWork>");
                await TaskEx.Delay(10);
                Log("</MoreWork>");
            }
        }
    }
    

    The output I would want is some interleaving of:

    1: <SomeWork>
    1 A: <MoreWork>
    1 A: </MoreWork>
    1 B: <MoreWork>
    1 B: </MoreWork>
    1: </SomeWork>

    with:

    2: <SomeWork>
    2 A: <MoreWork>
    2 A: </MoreWork>
    2 B: <MoreWork>
    2 B: </MoreWork>
    2: </SomeWork>

    But the stacks end up interfering with each other, and I end up with output like this (one example, from an actual run):

    1: <SomeWork>
    1 A: <MoreWork>
    1 A 2: <SomeWork>
    1 A 2 A: <MoreWork>
    1 A 2 A: </MoreWork>
    1 A 2: </MoreWork>
    1 A B: <MoreWork>
    1 A B B: <MoreWork>
    1 A B B: </MoreWork>
    1 A B B: </MoreWork>
    1 A: </SomeWork>
    1 A B: </SomeWork>

    I did some work on this a few weeks ago, and I ended up with these goals for an "async call context":

    • Flow automatically down sync and async invocations. If A() calls B(), then B() should inherit its async call context from A().
    • Flow all the way through an async method, even if it awaits ConfigureAwait(false). A() should have the same async call context when it resumes from an await, regardless of thread, SyncContext, or TaskScheduler.
    • Provide a hook for cloning to parallel async invocations. If A() starts B() and C() and then awaits Task.WhenAll, both B() and C() should get deep clones of A()'s async call context. (This is the hard part).
    • Implicit, so a mid-level async method may be unaware of it. If A() calls B() and B() calls C(), then B() may be written without any knowledge of the async call context.
    • Do not interfere with WCF/remoting calls. Async call contexts should not be serialized to other systems (or at least be able to opt-out of serialization).

    It's not clear how to implement this given the current tools (SyncContext/LogicalCallContext). It's certainly not clear how to do it efficiently.

    The most obvious application for this is an NDC stack for logging, but there are other applications, too. Impersonation comes to mind; I tried to get a basic proof-of-concept going with GenericPrincipal, but there's something wrong with the context flow under Async CTP, even with serial async code (I assume this is a known issue).

    BTW, Jon Skeet has a relevant post here.

           -Steve


    Programming blog: http://nitoprograms.blogspot.com/
      Including my TCP/IP .NET Sockets FAQ
      and How to Implement IDisposable and Finalizers: 3 Easy Rules
    Microsoft Certified Professional Developer

    How to get to Heaven according to the Bible

    Thursday, March 29, 2012 4:54 AM

All replies

  • I agree that Microsoft should revisit this and provide an extensible mechanism of some kind In the meantime, I think it would be fun to explore and discuss alternatives.

    FYI, I wrote the ConnectedProperties library, which may be a part of a solution.

           -Steve


    Programming blog: http://nitoprograms.blogspot.com/
      Including my TCP/IP .NET Sockets FAQ
      and How to Implement IDisposable and Finalizers: 3 Easy Rules
    Microsoft Certified Professional Developer

    How to get to Heaven according to the Bible
    Thursday, January 19, 2012 10:10 PM
  • ConnectedProperties looks good. Can it be used to connect properties to the current ExecutionContext? Haven't had a chance to look at the code yet, am assuming its more than just a static dictionary of objects ;)
    • Edited by ljwagerfield Thursday, January 19, 2012 11:36 PM
    Thursday, January 19, 2012 11:36 PM
  • ConnectedProperties can connect to any instance of a reference type. The "instance" part is where it gets interesting - you have to consider lifetime issues.

    ConnectedProperties handles the lifetime of the property values: they become eligible for GC as soon as the host instance is eligible for GC, even if the property value (directly or indirectly) contains a reference to the host instance. This is not possible with a dictionary of objects or even a dictionary with weak references for keys; ConnectedProperties is built on the only .NET Ephemeron, ConditionalWeakTable.

     

    So, yes, it can connect properties to an ExecutionContext; but that wouldn't be sufficient for a solution. If ExecutionContext.Capture returned a new instance each time it's called, then you couldn't connect properties to it and be able to retrieve them later. Also, when an ExecutionContext is copied (ExecutionContext.CreateCopy), the connected properties wouldn't carry over.

    There isn't an obvious solution (other than CallContext, which has its own problems). But ConnectedProperties can give us just one step, if we need it.

           -Steve


    Programming blog: http://nitoprograms.blogspot.com/
      Including my TCP/IP .NET Sockets FAQ
      and How to Implement IDisposable and Finalizers: 3 Easy Rules
    Microsoft Certified Professional Developer

    How to get to Heaven according to the Bible
    Friday, January 20, 2012 2:36 AM
  • Well, one option is to just use DI in a class wrapper.

    Asynchronous programming naturally encourages a functional programming style, rather than strictly object-oriented. If you embrace the functional approach, then you can encapsulate functional methods in an object that uses DI.

    e.g.:

    public sealed class MyAsyncOpContext
    {
      private ILogger logger; // DI
    
      public async Task<int> MyOpAsync()
      {
        logger.Log("begin");
        await ...;
        logger.Log("end: " + result);
        return result;
      }
    }
    

    There are some definite disadvantages to this approach:

    • There would be a tendency to group methods into objects by which dependencies they need, rather than by their functionality.
    • Calling async methods in other objects becomes awkward. i.e., "await new OtherContext().OpAsync()"

    Still, it's an option to consider.

           -Steve


    Programming blog: http://nitoprograms.blogspot.com/
      Including my TCP/IP .NET Sockets FAQ
      and How to Implement IDisposable and Finalizers: 3 Easy Rules
    Microsoft Certified Professional Developer

    How to get to Heaven according to the Bible
    Tuesday, January 24, 2012 2:29 PM
  • This SO question is related; log4net needs a context like this.

    I opened a suggestion on UserVoice where we can vote up this idea.

           -Steve


    Programming blog: http://nitoprograms.blogspot.com/
      Including my TCP/IP .NET Sockets FAQ
      and How to Implement IDisposable and Finalizers: 3 Easy Rules
    Microsoft Certified Professional Developer

    How to get to Heaven according to the Bible

    Tuesday, March 20, 2012 1:13 PM
  • Stephen;

      I think my first wife may have been a Epemeron...


    JP Cowboy Coders Unite!

    Tuesday, March 20, 2012 1:16 PM
  • As was mentioned earlier in the thread, .NET already provides CallContext.LogicalSet/GetData which can be used for this with async methods.  What additional functionality is being asked for here?  Stephen, what are the problems you refer to when you say "other than CallContext, which has its own problems"?  Thanks.


    Monday, March 26, 2012 3:39 PM
  • Stephen -

    Thanks for responding. I hope you don't mind a long post. :)

    Consider the SO question: the op is looking for a way to have an async-friendly "NDC" (Nested Diagnostic Context) - this is similar to CorrelationManager.LogicalOperationStack, but NDC is commonly used at a more granular level (so the example in this post is contrived but not unrealistic).

    Here's a simple MyNDCStack type with push/pop/print operations based on LogicalCallContext:

    public sealed class MyNDCStack
    {
        private const string name = "MyNDCStack";
        private readonly ConcurrentStack<string> stack;
    
        public MyNDCStack()
        {
            this.stack = new ConcurrentStack<string>();
        }
    
        public IDisposable PushContext(string context)
        {
            this.stack.Push(context);
            return new Pushed(this);
        }
    
        private void Pop()
        {
            string junk;
            this.stack.TryPop(out junk);
        }
    
        private static MyNDCStack CurrentContext
        {
            get
            {
                var ret = CallContext.LogicalGetData(name) as MyNDCStack;
                if (ret == null)
                {
                    ret = new MyNDCStack();
                    CallContext.LogicalSetData(name, ret);
                }
    
                return ret;
            }
        }
    
        public static IDisposable Push(string context)
        {
            return CurrentContext.PushContext(context);
        }
    
        public static string Current
        {
            get
            {
                return string.Join(" ", CurrentContext.stack.ToArray().Reverse());
            }
        }
    
        private sealed class Pushed : IDisposable
        {
            private MyNDCStack stack;
    
            public Pushed(MyNDCStack stack)
            {
                this.stack = stack;
            }
    
            public void Dispose()
            {
                if (this.stack != null)
                    this.stack.Pop();
                this.stack = null;
            }
        }
    }
    

    This works fine with serial/nested async operations, but it doesn't work so well when I use it with async operations running in parallel:

    class Program
    {
        static void Log(string message)
        {
            Console.WriteLine(MyNDCStack.Current + ": " + message);
        }
    
        static void Main(string[] args)
        {
            try
            {
                var tasks = new[]
                {
                    SomeWork("1"),
                    SomeWork("2"),
                };
    
                Task.WaitAll(tasks);
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex);
            }
            Console.ReadKey();
        }
    
        static async Task SomeWork(string stackName)
        {
            using (MyNDCStack.Push(stackName))
            {
                Log("<SomeWork>");
                await MoreWork("A");
                await MoreWork("B");
                Log("</SomeWork>");
            }
        }
    
        static async Task MoreWork(string stackName)
        {
            using (MyNDCStack.Push(stackName))
            {
                Log("<MoreWork>");
                await TaskEx.Delay(10);
                Log("</MoreWork>");
            }
        }
    }
    

    The output I would want is some interleaving of:

    1: <SomeWork>
    1 A: <MoreWork>
    1 A: </MoreWork>
    1 B: <MoreWork>
    1 B: </MoreWork>
    1: </SomeWork>

    with:

    2: <SomeWork>
    2 A: <MoreWork>
    2 A: </MoreWork>
    2 B: <MoreWork>
    2 B: </MoreWork>
    2: </SomeWork>

    But the stacks end up interfering with each other, and I end up with output like this (one example, from an actual run):

    1: <SomeWork>
    1 A: <MoreWork>
    1 A 2: <SomeWork>
    1 A 2 A: <MoreWork>
    1 A 2 A: </MoreWork>
    1 A 2: </MoreWork>
    1 A B: <MoreWork>
    1 A B B: <MoreWork>
    1 A B B: </MoreWork>
    1 A B B: </MoreWork>
    1 A: </SomeWork>
    1 A B: </SomeWork>

    I did some work on this a few weeks ago, and I ended up with these goals for an "async call context":

    • Flow automatically down sync and async invocations. If A() calls B(), then B() should inherit its async call context from A().
    • Flow all the way through an async method, even if it awaits ConfigureAwait(false). A() should have the same async call context when it resumes from an await, regardless of thread, SyncContext, or TaskScheduler.
    • Provide a hook for cloning to parallel async invocations. If A() starts B() and C() and then awaits Task.WhenAll, both B() and C() should get deep clones of A()'s async call context. (This is the hard part).
    • Implicit, so a mid-level async method may be unaware of it. If A() calls B() and B() calls C(), then B() may be written without any knowledge of the async call context.
    • Do not interfere with WCF/remoting calls. Async call contexts should not be serialized to other systems (or at least be able to opt-out of serialization).

    It's not clear how to implement this given the current tools (SyncContext/LogicalCallContext). It's certainly not clear how to do it efficiently.

    The most obvious application for this is an NDC stack for logging, but there are other applications, too. Impersonation comes to mind; I tried to get a basic proof-of-concept going with GenericPrincipal, but there's something wrong with the context flow under Async CTP, even with serial async code (I assume this is a known issue).

    BTW, Jon Skeet has a relevant post here.

           -Steve


    Programming blog: http://nitoprograms.blogspot.com/
      Including my TCP/IP .NET Sockets FAQ
      and How to Implement IDisposable and Finalizers: 3 Easy Rules
    Microsoft Certified Professional Developer

    How to get to Heaven according to the Bible

    Thursday, March 29, 2012 4:54 AM
  • Thanks for the thorough response, Steve.  It's helpful to have a better understanding of what you're looking for.
    Friday, March 30, 2012 1:53 PM