none
Anonymous Delegate lifetime scope RRS feed

  • Question

  • If one defines an anonymous delegate, say, inside a function that uses a local variable of that function but the anonymous method can be called outside the function(say it is added to a list) then how is the local variable used by the anonymous delegate?

     

    A test shows that the variable does have the correct value which I assume is because it is not GC'ed. It also showed that the variable has the last value(right before the function call ended rather than the before(which is generally how you think about it)).

     

    This seems highly error prone but I guess is acceptable.

     

    But how is the stack itself for the function defined? Is the function not GC'ed either?

    It seems one has to be extremely careful when using non-local variables inside the definition of an anonymous function. As far as I can tell it is useless to do so(Since it is outside the scope of the function) and an error warning show be thrown?

    (Note I am not talking about calling an anonymous function but defining it)
    Saturday, March 26, 2011 2:08 AM

All replies

  • Anonymous functions are syntactic sugar, the compiler generally turns them into proper named functions.  In the case you mention, the compiler will generally go a step further and create a class with your function and a member variable to store the data it needs for the lifespan of the delegate object.  For instance, if you investigate the IL for the code generated from the following simple program.

    class Program
    {
      static void Main(string[] args)
      {
        Func1();
        Func2();
        Func2();
      }
    
      delegate void SimpleDelegate();
      static SimpleDelegate CallMeLater;
    
      static void Func1()
      {
        int q = 42;
        CallMeLater = new SimpleDelegate(delegate()
        {
          Console.Write(q.ToString() + " - ");
          q += 1;
          Console.WriteLine(q.ToString());
        });
      }
    
      static void Func2()
      {
        CallMeLater();
      }
    }
    

    Will actually create code that looks closer to the following code block (though, in reality the names used by the compiler for the generated functions and classes are names that aren't valid in C# to prevent conflicts with names you'd use):

    class Program
    {
      static void Main(string[] args)
      {
        Func1();
        Func2();
        Func2();
      }
    
      delegate void SimpleDelegate();
      static SimpleDelegate CallMeLater;
    
      private sealed class delegateHelperClass
      {
        public int q;
    
        public void delegateFunction()
        {
          Console.Write(this.q.ToString() + " - ");
          this.q++;
          Console.WriteLine(this.q.ToString());
        }
      }
    
      static void Func1()
      {
        int q = 42;
        delegateHelperClass helper = new delegateHelperClass();
        helper.q = q;
        CallMeLater = helper.delegateFunction;
      }
    
      static void Func2()
      {
        CallMeLater();
      }
    }
    
    Saturday, March 26, 2011 6:58 AM
  • um, it may or may not do that but your example is not totally correct. Add q = 10 after the delegate definition then q will be 51 in the delegate call. That is, it uses the last value of the function rather than what it was in place.

     

     static void Func1()
     {
      int q = 42;
      CallMeLater = new SimpleDelegate(delegate()
      {
       Console.Write(q.ToString() + " - ");
       q += 1;
       Console.WriteLine(q.ToString());
      });
      q = 10; // or assign a random value
     }
    

    then 10 will be passed to the delegate instead(unless the anonymous function is called inside Func1()) and not 42 which you may think. What I think is going on is that a reference is held to q so it is not cleaned up. Now suppose you call Func1 more than once with q randomly assigned? What is the value of q?

     

    This can result in unpredictable behavior in complex routines since the variables used may not be one one expects. It should throw a warning at least if a non-local variable is used.

     

    Basically the non-local variables act as global variables to the anonymous function yet local to the function that contains the definition. To do this the variable actually must be defined globally(or perpetuated on the stack) and effectively is a global variable that has local scope. This, IMO can cause some major problems. I'm generally not in favor of making something more restrictive but in this case at the very least we need a warning for it.

     

    Saturday, March 26, 2011 3:29 PM
  • um, it may or may not do that but your example is not totally correct. Add q = 10 after the delegate definition then q will be 51 in the delegate call. That is, it uses the last value of the function rather than what it was in place.

    Sure.  If the compiler notices that you change that variable, it will move the assignment of the class's copy of the variable to after the change.  In other words, in my example the imaginary "helper.q = q;" line will be moved to after the "q = 10;" line.

    You can verify all of this yourself with ildasm, or by perusing the C# spec.

    My point in all of this is that there's not really any GC magic to anonymous functions, the function containing the anonymous function is available for normal GC rules since the anonymous function is implemented as a normal function, or if necessary, a class.

    If you feel strongly that the compiler should issue a warning if you use the outer variables, then I'd suggest you raise a bug with Microsoft Connect.


    Saturday, March 26, 2011 4:14 PM
  • I'm sorry, your wrong. If it was as simple as you said then the example using a non-deterministic variable wouldn't make any sense. If it was simply copying the result at the declaration of the function(if the variable was truly local as you say and was "copied" then it would have a static nature).

     

    Try an example by letting q be randomly determined after the assignment of the delegate. Print out q inside the delegate. If q was copied then the delegate would always print out the same value. It does not.

    Saturday, March 26, 2011 9:18 PM
  • Right, the assignment to the compiler generated class happens after the last change to the outer variable.

    Sunday, March 27, 2011 1:02 AM
  • Hi Jon,

     

    I am writing to check the status of the issue on your side. Would you mind letting us know the result of the suggestions?

     

    If you have got answers, please remember to mark answer and close this thread.

     

    Have a nice day!


    Paul Zhou [MSFT]
    MSDN Community Support | Feedback to us
    Get or Request Code Sample from Microsoft
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Monday, April 4, 2011 2:18 AM
  • The following code shows the problem. x, which is used in the action, is not passed the the x at the action declaration but basically a reference to x. When the action is eventually called it uses the reference to x, which is not GC'ed because it is being referenced. But x's "scope" as long passed as far as most would consider since it was defined locally to the function... yet somehow exists globally with respect to the action.

     

    I think most will think that the action uses the value x when it is declared just as cowered thought. This is completely wrong. Also most think x will be GC'ed at the end of the function. Again this is wrong because the action holds a reference to x. The value that the action actually gets is the value at the end of the function(assuming it is local to the function)

     

    The only place such a thing would work is if the anonymous method is called in the same scope as the variables it uses AND the variable is not changed in between that time. This is sometimes useful. In any case I think a warning should be thrown since in complex cases it could be a very hard bug to find.

     

    The real problem is mainly what one thinks the value of x really is in the example. I think most people will say it is 10. But as the code shows it is random on the first call of the action.

     

    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    
    namespace AnonymousFunctionTest
    {
    
    	class Program
    	{
    		static Random r = new Random();
    		static object xref = null;
    		static Action action = null;
    		static void Main(string[] args)
    		{
    			Test();
    			action();
    			action();
    			Test();
    			Test();
    			action();
    			action();
    
    			Console.ReadKey();
    
    		}
    		
    		
    		static void Test()
    		{
    			var x = 10;
    			if (action == null)
    				action = () =>
    					{
    						Console.WriteLine("FC = " + x);
    						x = -1;
    						xref = x;
    					};
    
    
    			x = r.Next(0, 1000);
    			Console.WriteLine("x = " + x);
    			
    		}
    	}
    }
    
    

     

     

     

     

     

     

    Monday, April 4, 2011 11:47 AM
  • Hi Jon,

     

    The local variables and parameters whose scope contains an anonymous method declaration are called outer variables of the anonymous method.

    Unlike local variables, the lifetime of the captured variable extends until the delegates that reference the anonymous methods are eligible for garbage collection.

    So in your case, the x is called outer variable. And your anonymous method Action will reference the first x and the lifetime of x will keep until Action is collected. So before Action is collected, when you call Action every time, you will get the value of the first x.

     

    Much further information, please refer to:

    Anonymous Methods (C# Programming Guide)


    Paul Zhou [MSFT]
    MSDN Community Support | Feedback to us
    Get or Request Code Sample from Microsoft
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Friday, April 8, 2011 6:15 AM
  • It doesn't matter what you call them, it seems to me it is error prone either way. They effectively behave like global variables to the anonymous function yet local variables to the function declaration. This is logically ambiguous(although programmatically is dealt with in a non-ambiguous way).

     

    A simple fix would be to require that such "outer variables" be unused outside the anonymous function(although they could be initialized at declaration). This seems a lot safer and just as good.

     

     

    Friday, April 8, 2011 6:28 AM
  • This is because the lifetime of Action decides the lifetime of x.

    If you modify the code as below:

    static Random r = new Random();

            static object xref = null;

            //static Action action = null;

            static void Main(string[] args)

            {

                Test();

                Test();

                Test();

                Test();

                Test();

                Console.ReadKey();

            }

     

            static void Test()

            {

                var x = 10;

                Action action = () =>

                    {

                        xref = x;

                        Console.WriteLine("FC = " + x);

                        x = -1;

                        xref = x;

                    };

                action();

                x = r.Next(0, 1000);

                Console.WriteLine("x = " + x);

            }

    When action is collected, the x will be freed too. In your original code, there is only one “x”(the first x) will be alive until the action is collected.

    In above code, the lifetime of action is only within the scope of Test(). You can see that action and x are  freed after Test completed. Every time you call Test(), x will be 10.


    Paul Zhou [MSFT]
    MSDN Community Support | Feedback to us
    Get or Request Code Sample from Microsoft
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Friday, April 8, 2011 7:50 AM
  • Yes, I understand that. It's because the anonymous function has a reference to x and so x will not be garbage collected. That is not the point. The point is the value that x contains inside the anonymous function. Because it may not be the value that one thinks it will be it could lead to problems. The above examples are too simple to show any real issues but in real world problems it might be totally different.

     

    The point is, that because the way "outer variables" are handled in anonymous functions there is no real need to have them except to have global variables(or static w.r.t. the anonymous function) but this can be achieved much safer by simply requiring them to be unused in the "outer function"

    Action nuker;

    void EndOfWorld(bool SendNukes)

    {

    var x = SendNukes;

    nuker = () => { if (x) SendTheNukes(); }

    x = true;

    }

     

    What is the behavior of this function? As far as action is concerned x is always false and SendTheNukes() is always executed. BUT this doesn't seem to be the logic intended by the programmer since he wouldn't even use the *if* statement if that were the case.

     

    So when ever nuker is called we will simply be calling SendTheNukes. The problem is that in nuker we are using x as an global variable but in EndOfWorld x is local. Since x is local in EndOfWorld it's value is determined at the end of EndOfWorld(which in this case is true) which means that, as a global variable w.r.t. to nuker(or an outer variable), x's initialization is true and not SendNukes like we think it should be.

     

    This is not much different than using global or static variables with normal functions.

     

    static bool x = false;

    void nuker() { if (x) SendTheNukes(); }

     

    // somewhere

    x = true

     

    Whats the difference? The above case always has x == false until x is changed to true "external" to nuker. If we call nuker for the first time it will have x == false. But in the anonymous delegate x == true all the time. This is because in the above case the "outer function" would be the program which doesn't stop execution. In the anonymous function version the "outer function" is EndOfWorld which returns quickly(and hence effecting the value of x).

     

    Anyways, to me it is a bit error prone and I don't see any need for it to be the way it is since.

     

    void EndOfWorld(bool SendNukes)

    {

    var x = SendNukes;

    nuker = () => { if (x) SendTheNukes(); }

    x = true;

    }

     

    is identical to

     

    void EndOfWorld(bool SendNukes)

    {

    var x = SendNukes;

    x = true;

    nuker = () => { if (x) SendTheNukes(); }

    }

     

    but is entirely different than

     

    void EndOfWorld(bool SendNukes)

    {

    var x = SendNukes;

    if (x) SendTheNukes();

    x = true;

    }

     

     

    NOTE: The above code doesn't actually call nuker. You have to realize that it takes a more complex program to really give problems. There's nothing wrong with the way C# handles it. We just need to prevent ourselves from ever thinking that x maybe something it isn't(and because of out they are used it doesn't hurt anything to do so).

    Coward is proof that one can get it wrong.  I'm sure I've gotten it wrong. In some cases it won't show up because the outer variable is not changed below the definition of the anonymous function.

     

    In fact what we need to do is remove all outer variables from anonymous method declarations and have the ability to create static variables for them. This would prevent any problems with such things. Those static variables would have the same lifetime as the anonymous method.

     

     

     

     

     

     

     

     

    Friday, April 8, 2011 10:08 PM
  • Hi JonSlaughter again,

     

    Just my opinion: I think outer variables are safer than global variables.

    At least, global variables can be changed everywhere until the process is realeased. But outer variables can be only changed in the method body.

     

    In your code, we can see that nuker will be defined in EndOfWorld method. And the x can only be changed in the method EndOfWorld. Out of the method EndOfWorld, we can not find the variable “x” except delegate “nuker”. If we define an outer variable that used in anonymous method, we can change its value in the scope of the delegate. If we define a global variable “x” and use it in the anonymous method, the “x” may be changed at other places. So I think that it’s better to say that we decide to use an outer variable or global variable depends on requirement instead of saying that global variable is better than outer variable.

     

    What I want to say is that differet requirement decides different chosen.

     


    Paul Zhou [MSFT]
    MSDN Community Support | Feedback to us
    Get or Request Code Sample from Microsoft
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Monday, April 11, 2011 3:42 AM
  • Yes, but that is also precisely the problem. When the function ends what happens to x? It is held onto by the anonymous function and used when it is called. The time that passes between this may be very long and the programmer may not realize what is going on. A warning can alert him that the variable may hold a value that wasn't intended. One can think of an example of a handle in win32. Suppose you allocate a handle and used in an anonymous function(as an outer variable) that is actually called in a thread(defined in some function and passed to to a thread pool or whatever). The handle is closed at the end of the function. If the thread executes after the function is finished then the handle will be invalid. This could be a potentially difficult problem to detect. A warning could easily be thrown that one is using an outer variable in the declaration of an anonymous function which may help locate the problem.

     

    It is much safer to give a warning about their use. OTOH, if the delegate is being called immediately then no harm should be done and it can be useful to use outer variables.

     

     

    Monday, April 11, 2011 11:23 AM
  • ·        Consider the following example :

    ·       

    {
     int q = 42;
     CallMeLater = new SimpleDelegate(delegate()
     {
      Console.Write(q.ToString() + " - ");
      q += 1;
      Console.WriteLine(q.ToString());
     });
     q = 10; // or assign a random value
     }

     

    What we really trying to create isn’t merely a delegate but also creating a closure. The outer variable q becomes captured, and therefore is part of the state of the resulting closure. q is not ‘owned’ by the delegate though, so the Func1 still can change q’s value if it wants to.

     

    The reason for this behavior is that the anonymous method is a closure and is bound to its parenting method body and the local variables in it. The important distinction is that it is bound to variables, not to values. In other words, the value of "q" is not copied in when "CallMeLater" is declared. Instead, a reference to "q" is used so that "CallMeLater" will always use the most recent value of "q". In fact, this reference to "q" will be persisted even if "q" goes out of scope.

     

    Everything above is handled by the compiler and there is no runtime support. This is done by the compiler by creating a special helper class.

    To know how this is done refer the following link :

    http://blogs.msdn.com/b/oldnewthing/archive/2006/08/02/686456.aspx

     

    Let's look at an example in which using global variables(or static w.r.t. the anonymous function) won’t work:

    static List<string> g_Names = new List<string>(
    new string[] {
    "Bill Gates",
    "Dustin Campbell",
    "Dustin's Trophy Wife",
    "Foster the Dog"
    });

    static void Print(List<string> items)
    {
    foreach (string item in items)
    Console.WriteLine(item);
    }

    static void Main(string[] args)
    {
    Print(g_Names);
    }

    This application simply creates a list of names and then outputs them to the console. It works perfectly well.

    Now, let's assume that we need to add the ability to retrieve a list of names that starts with some particular text. This is pretty easy to implement because there is a handy method on List<T> called FindAll that simply takes a Predicate delegate and produces a new List<T> containing all of the items that the Predicate returns true for. We can add this new function like so:

    static string g_StartingText;

    static bool NameStartsWith(string name)
    {
    return name.StartsWith(g_StartingText);
    }

    static List<string> GetNamesStartingWith(string startingText)
    {
    g_StartingText = startingText;
    return g_Names.FindAll(NameStartsWith);
    }

    Everything is working fine until our client calls and says that there is a new requirement for this function: it must be thread-safe. In other words, the function must produce valid results even if it is called by multiple threads. This is problematic because, while one thread is finding all names starting with "D", another thread could change "g_StartingText" to something else and bad results would be returned.

    One possibility might be tempting to place a lock on "g_StartingText". This would certainly make the function thread-safe but it has some drawbacks. The biggest issue with this approach is that threads will not be able to access this function concurrently. If a thread acquires the lock, all other threads must wait until that thread is finished. In other words, this method becomes a potential bottleneck because only one thread can access it at a time and if there any additional processors on the machine they won't be used.

    The solution is to use an anonymous method to create a closure and remove the shared state:

    static List<string> GetNamesStartingWith(string startingText)
    {
    return g_Names.FindAll(
    delegate(string name)
    {
    return name.StartsWith(startingText);
    });
    }

     

    Even with the verbosity of anonymous methods, the code has been greatly reduced. And, assuming that "g_Names" will never be modified, this function could run concurrently on multiple threads and multiple cores without any synchronization.

     

    While outer variable changed by the anonymous function might not what you want in a particular instance, in a different context, you might want the variable to be modifiable after the delegate has been created.

     

    To answer, why doesn’t Visual Studio warn you? I suspect the answer is that the code given is syntactically correct. Closures are a feature of the language and the expectation is that the developer will know what they are doing.


    --Trevor H.
    Send files to Hotmail.com: "MS_TREVORH"
    Wednesday, April 27, 2011 4:48 PM
    Moderator
  • "To answer, why doesn’t Visual Studio warn you? I suspect the answer is that the code given is syntactically correct. Closures are a feature of the language and the expectation is that the developer will know what they are doing."

     

    Oh come on... many of the features of C# are to remove the burden of having to be a great programmer. Whats the point of all the automatic garbage collection? If the "developer" knows what he is doing then he surely knows he has to release memory after he allocated it? 

     

    I think the real point is to prevent issues from things that can easily be overlooked but hard to find. This is one of those issues. 99% of the time there will be no real problem but that 1% when it's wrong could be extremely difficult to diagnose.

     

    In any case you won't be able to use an "inline" example to exhibit any problems since the anonymous function will always be called immediately. It's when an outer variable is used by an anonymous function and that function is called outside the scope the variable was declared in. Most simple examples will not exhibit any problems so a simple example won't prove anything.


    Thursday, April 28, 2011 4:34 PM
  • This problem is subject of debate amongst most of the developers for a long time.

     

    “A warning could easily be thrown that one is using an outer variable in the declaration of an anonymous function which may help locate the problem.”

     

    I respect your opinion and suggest you to open up an incident case with us if you think that you want this feature included in  Visual Studio as it is impacting your business. We can then contact the product group and propose this to them. They will look at factors like severity, business impact etc. and based on this may include this feature in the future version. Please visit this link to see the various support options that are available to better meet your needs:  http://support.microsoft.com/default.aspx?id=fh;en-us;offerprophone.

     


    --Trevor H.
    Send files to Hotmail.com: "MS_TREVORH"
    Monday, May 2, 2011 8:47 PM
    Moderator