locked
Inconsistent behaviour of Batch .NET API RRS feed

  • Question

  • I'm using the .NET Batch API, both from a console program and from an Azure web role. I put my basic functionality into it's own class library, so I get exactly the same behaviour in both cases, yet everything works fine on the console, but when I try to connect from the web role, a simple call to ListPools just hangs forever without throwing any errors.

    I've been trying to figure out what's going on for two days now, so I did try a lot of things...

    I'm definitely using the right http targets (https://batch.core.windows.net); any modification to that will throw the correct exceptions. I'm also using the right credentials; BatchClient.Connect doesn't complain and I get authenticated on the console.

    I've trimmed it down to a very basic example that I thought would have to work, yet it still hangs when called from the web role:

    BatchCredentials creds = new BatchCredentials(myname, mykey); IBatchClient bc = BatchClient.Connect("https://batch.core.windows.net", creds); IPoolManager pm = bc.OpenPoolManager(); IEnumerable<ICloudPool> pools = pm.ListPools(); List<ICloudPool> poolsl = pools.ToList(); // <-- hangs forever. // so does // foreach (ICloudPool p in pm.ListPools()) // and // IEnumerable<ICloudPool> pools = pm.ListPools(); // if (!pools.Select(pool => pool.Name).Contains(PoolName)) ...

    Does anybody have any idea what the reason for this could be? Also, is there a simple way I could get a look at the raw connection data, i.e., the REST GET/PUTs?

    Thanks!

    Christoph

    Tuesday, December 9, 2014 6:41 PM

Answers

  • Authors of C# in a message pumping context should study the use of the captured SynchronizationContext during transitions from asynchrony to synchrony (your .ToList() above).  One such article can be found here: C#: Avoiding Deadlock in Async/Await

    The captured SynchronizationContext deadlock must be mitigated when using ANY synchronous wrapper and those in Microsoft.Azure.Batch.dll version 1.0.7198.409 are not currently hardened for use in message pumping contexts (but hardening is in the pipeline, see below).

    When IEnumerableAsyncExtended<T> is used as IEnumerable<T> (like your .ToList() above), consumers of IEnumerable<T> will result in calls to MoveNext()... which is a synchronous wrapper around MoveNextAsync(), and succumbs to deadlock. 

    There are many articles out there that can guide you choosing a blocking mechanism best suited to your code.

    The most common recommendation is to use asynchrony instead of blocking calls.  This is considered best practice. 

    For IEnumerableAsyncExtended<T> there is IAsyncEnumerable<T>.

    If you must transition to synchrony, then I recommend studying something like Task.Run() and the resulting implications with respect to thread use, etc.

    Microsoft.Azure.Batch.dll version 1.0.7198.409 does not call ConfigureAwait(false) throughout its code paths.  Thus, all message pumping contexts that choose to transition to synchrony must use something like Task.Run().  Note that this holds for all synchronous wrappers AND direct use of the tpl task (.Wait or .Result).

    ConfigureAwait(false) on all code paths in the Azure Batch library is now in Test and will be available in the next release.  However, if your (ui/aspx/etc) code is using synchronous programming models now then care should be taken to avoid producing the deadlock organically.  The Azure Batch assembly will not use the captured context for its continuations but other code/assemblies may do so.




    daryl





    • Edited by DarylMsft Tuesday, December 9, 2014 10:02 PM hope
    • Marked as answer by Wintersteiger Wednesday, December 10, 2014 4:41 PM
    Tuesday, December 9, 2014 9:40 PM
  • The synchronize call in the current version of Batch Client library (v1.0.0) will deadlock the message pumping thread - that means WinForm/WPF/ASP.NET event handler. This behavior will be changed soon, in our next release.

    For now, you have 2 options. The first is to switch to async version of API. For example, the ToList() involves synchronize REST API call, so you can change the code to something like below. Note that your event handler should be marked as async too (see http://www.asp.net/web-forms/overview/performance-and-caching/using-asynchronous-methods-in-aspnet-45)

                var pools = pm.ListPools();
                IAsyncEnumerator<ICloudPool> en = pools.GetAsyncEnumerator();
                while (await en.MoveNextAsync())
                {
                    this.Log(en.Current.Name);
                }

    The second solution is to wrap the code in a new thread, e.g.,

    Task.Factory.StartNew(() => {
        IEnumerable<ICloudPool> pools = pm.ListPools();
        List<ICloudPool> poolsl = pools.ToList();
    });

    • Marked as answer by Wintersteiger Wednesday, December 10, 2014 4:41 PM
    Tuesday, December 9, 2014 11:05 PM

All replies

  • Authors of C# in a message pumping context should study the use of the captured SynchronizationContext during transitions from asynchrony to synchrony (your .ToList() above).  One such article can be found here: C#: Avoiding Deadlock in Async/Await

    The captured SynchronizationContext deadlock must be mitigated when using ANY synchronous wrapper and those in Microsoft.Azure.Batch.dll version 1.0.7198.409 are not currently hardened for use in message pumping contexts (but hardening is in the pipeline, see below).

    When IEnumerableAsyncExtended<T> is used as IEnumerable<T> (like your .ToList() above), consumers of IEnumerable<T> will result in calls to MoveNext()... which is a synchronous wrapper around MoveNextAsync(), and succumbs to deadlock. 

    There are many articles out there that can guide you choosing a blocking mechanism best suited to your code.

    The most common recommendation is to use asynchrony instead of blocking calls.  This is considered best practice. 

    For IEnumerableAsyncExtended<T> there is IAsyncEnumerable<T>.

    If you must transition to synchrony, then I recommend studying something like Task.Run() and the resulting implications with respect to thread use, etc.

    Microsoft.Azure.Batch.dll version 1.0.7198.409 does not call ConfigureAwait(false) throughout its code paths.  Thus, all message pumping contexts that choose to transition to synchrony must use something like Task.Run().  Note that this holds for all synchronous wrappers AND direct use of the tpl task (.Wait or .Result).

    ConfigureAwait(false) on all code paths in the Azure Batch library is now in Test and will be available in the next release.  However, if your (ui/aspx/etc) code is using synchronous programming models now then care should be taken to avoid producing the deadlock organically.  The Azure Batch assembly will not use the captured context for its continuations but other code/assemblies may do so.




    daryl





    • Edited by DarylMsft Tuesday, December 9, 2014 10:02 PM hope
    • Marked as answer by Wintersteiger Wednesday, December 10, 2014 4:41 PM
    Tuesday, December 9, 2014 9:40 PM
  • The synchronize call in the current version of Batch Client library (v1.0.0) will deadlock the message pumping thread - that means WinForm/WPF/ASP.NET event handler. This behavior will be changed soon, in our next release.

    For now, you have 2 options. The first is to switch to async version of API. For example, the ToList() involves synchronize REST API call, so you can change the code to something like below. Note that your event handler should be marked as async too (see http://www.asp.net/web-forms/overview/performance-and-caching/using-asynchronous-methods-in-aspnet-45)

                var pools = pm.ListPools();
                IAsyncEnumerator<ICloudPool> en = pools.GetAsyncEnumerator();
                while (await en.MoveNextAsync())
                {
                    this.Log(en.Current.Name);
                }

    The second solution is to wrap the code in a new thread, e.g.,

    Task.Factory.StartNew(() => {
        IEnumerable<ICloudPool> pools = pm.ListPools();
        List<ICloudPool> poolsl = pools.ToList();
    });

    • Marked as answer by Wintersteiger Wednesday, December 10, 2014 4:41 PM
    Tuesday, December 9, 2014 11:05 PM