Why async methods cannot have ref or out parameters?

Question

When I tried to convert some of my methods to async I realized that async methods do not support ref or out params (similar to iterators).

I found a workaround for that by creating a dedicated class to holding all the ref and out params of the async method and passing an instance of that object to the method instead of the ref or out parmas. The async method will change the members of this instance object and by that act as if the object members where 'ref' or 'out'.

After the async method is awaited, I retrived the values from the instance object and continue my logic.

For example, the following sync method:

public class SyncClass { 
  public void FunctionX(ref int param1, ref string param2, out object param3){
   param1 = 10;
   param2 = "string";
   param3 = new StringBuilder("string builder");
   // .. syncronous operation
  }
}

 

will tranform into the following async method:

public class AsyncClass { 
  public class Out_Ref_Params {
   public int param1;
   public string param2;
   public object param3;
  }

  public async Task FunctionXAsync(Out_Ref_Params outRefParams) {
   outRefParams.param1 = 10;
   outRefParams.param2 = "string";
   outRefParams.param3 = new StringBuilder("string builder");
   // .. await asyncronous operation;
  }  
}

and inside the calling method:

public class Program { 
 private async static void Main(string[] args) { 
  AsyncClass x = new AsyncClass();
 
  // Declare variables to be pass by ref/out
  int i = 5;
  string s = "s";
  object o = null;
  // Save params to object
  AsyncClass.Out_Ref_Params outRefParams = new AsyncClass.Out_Ref_Params{param1=i,param2=s,param3=o};
  await x.FunctionXAsync(outRefParams);

  // Retrive new values/references
  i = outRefParams.param1;
  s = outRefParams.param2;
  o = outRefParams.param3;

  // .. continue program logic
 }
} 

 

So my question is why not let the compiler do all the hard work for us? (like it already does so wonderfully for making the method async in the first place).

(By the way: I would like to thank all those who brought us this wonderful new feature, It makes life so much easier)

Thanks,

Yaron

Answer 1

r

Answer 2

Hi Yaron.

You've identified one workaround to "out" parameters. The other workaround is for the async method to return a tuple.

As for why we can't do this implicitly? The only possible way would be for the compiler to implicitly generate this class, and then "copy-out" the contents of the class into the out parameters once the async method had finished.

Incidentally, VB already uses this "copy-out" pattern for its ref parameters under certain circumstances e.g. if you passed a property for a ByRef parameter. It gets confusing for the VB developer to figure out under what circumstances a ByRef parameter is handled by reference, and under what circumstance it's handled by pointer.

While it's been used sometimes VB, this copy-out behavior would definitely be a break with precedent for C#. We think that people have come to expect that C# out parameters are pretty much equivalent pointers in C++. The behavior expected of C# is that the change to the out parameter is visible AS SOON AS the assignment has been made.

What kind of code would demonstrate the difference by out-by-reference and out-by-copyout? imagine this...

bool shouldDisableControls = false;
await DoWorkAsync(out shouldDisableControls);
if (shouldDisableControls) Button1.Enabled = false;
...

async Function DoWorkAsync(out shouldDisableControls)
{
    // this is the critical bit where the button has to be disabled
    shouldDisableControls = true;
    await Database.FetchData(); // might or might not return control to caller
    // This next bit isn't critical
    shouldDisableControls = false;
    await Task.Sleep(1000);
}

If we did out-by-reference then the code would work, i.e. the button would be disabled during the call to FetchData. If we did out-by-copyout then it wouldn't be disabled.

 

Note also that out-by-copyout would have to be performed at the callsite, as it is in your case. In general the callsite has no knowledge of whether the method it's calling was marked with the "async" modifier or was just a regular old method that returned Task... the modifier is purely an implementation detail. So if we were to make this change, it'd have to apply to all awaits of invocations of awaitable functions. It would need some magic to tie together patterns like
  int x;
  var t = FredAsync(out x);
  ...
  await t; // presumably the copy-out happens here

And I don't see any general way to know when the caller should do copy-out.

 

As for why async methods don't support out-by-reference parameters? (or ref parameters?) That's a limitation of the CLR. We chose to implement async methods in a similar way to iterator methods -- i.e. through the compiler transforming the method into a state-machine-object. The CLR has no safe way to store the address of an "out parameter" or "reference parameter" as a field of an object. The only way to have supported out-by-reference parameters would be if the async feature were done by a low-level CLR rewrite instead of a compiler-rewrite. We examined that approach, and it had a lot going for it, but it would ultimately have been so costly that it'd never have happened.

--
Lucian Wischik, VB language PM

 

Answer 3

I know this is many months later, but I thought my input might other people who come across this discussion who are trying to solve similar design problems.

I have another idea for a work-around, using closures. For example if the parameters are only "out" parameters the async function could look like this:

public async Task FunctionXAsync(Action<int> setParam1, Action<string> setParam2, Action<object> setParam3) 
{
  // .. await asyncronous operation;
  setParam1(10);
  setParam2("string");
  setParam3(new StringBuilder("string builder"));
  // .. await asyncronous operation;
} 

And then the caller can bind the variables using lambda expressions like this:

// Declare variables to be pass by ref/out
int i;
string s;
object o;

await FunctionXAsync(value => i = value, value => s = value, value => o = value);

If you need "ref" parameters, you can do the same thing with getters as well:

async static void Func(Action<int> setParam1, Func<int> getParam1)
{
  if (getParam1() != 1)
    setParam1(10);
}

...
static void Test()
{
  int i = 0;
  await Func(p => i = p, () => i);
}

This method has the advantage that setting (or getting) the out/ref arguments will set the original variable in real-time, and it also works with properties. There is no "copy-out" that needs to be done at some ill-defined time - calling the setter makes the output "visible as soon as the assignment is made".

An obvious disadvantage is the readability, but in some cases it might be better than using custom objects or tuples. Another disadvantage might be the performance hit (although I haven't tested) from binding each "argument" variable to at least one separate closure (a getter and/or setter) - whether this is a problem or not depends on the context. Another disadvantage of this approach (or the parameters-object workaround) is that you loose the compiler warnings that tell you when you've neglected to set an out parameter, or when the caller is using an unset variable etc - this would not be a problem when using tuples.

As an extra note, since the async function is actually accessing the original variables, in each particular situation I would take careful consideration about multiple-access issues that might crop up you have multiple async functions accessing the same bound variables - especially (but not only) if they might be scheduled on different threads.

If the CLR were ever to support ref/out parameters on async functions, I would expect it to somehow implicitly bind the arguments to a closure of sorts if it were to keep the C# way that changes to variables are visible immediately.

Hope this solution helps somebody!

 

Mike