none
How to create async method which encapsulates WebRequest, BeginGetRequestStream, BeginGetResponse, BeginWrite, BeginRead

    Question

  • How to create an async function which calls several async methods sequentially and then returns Task<T>? Here is a mockup:

      public static Task<string> FetchAsync()
      {
        string url = "http://www.snee.com/xml/crud/posttest.cgi", message = "Hello World!";
    
        var request = (HttpWebRequest)WebRequest.Create(url);
        request.Method = WebRequestMethods.Http.Post;
    
        return Task.Factory.FromAsync<Stream>(request.BeginGetRequestStream, request.EndGetRequestStream, null)
          .ContinueWith(t =>
          {
            var stream = t.Result;
            var data = Encoding.ASCII.GetBytes(message);
            Task.Factory.FromAsync(stream.BeginWrite, stream.EndWrite, data, 0, data.Length, null, TaskCreationOptions.AttachedToParent)
              .ContinueWith(t2 => { if (stream != null) stream.Close(); });
          })
          .ContinueWith<string>(t =>
          {
            var t1 =
              Task.Factory.FromAsync<WebResponse>(request.BeginGetResponse, request.EndGetResponse, null)
              .ContinueWith<string>(t2 =>
              {
                using (var response = (HttpWebResponse)t2.Result)
                using (var stream = response.GetResponseStream())
                {
                  var buffer = new byte[response.ContentLength];
                  var t3 = Task<int>.Factory.FromAsync(stream.BeginRead, stream.EndRead, buffer, 0, buffer.Length, null, TaskCreationOptions.AttachedToParent)
                    .ContinueWith<string>(t4 =>
                    {
                      return Encoding.ASCII.GetString(buffer);
                    });
                  t3.Wait();
                  return t3.Result;
                }
              });
            t1.Wait();
            return t1.Result;
          });
      }

    Please advise how to implement it properly. It should return Task<string>, send HTTP POST request with some data, return a result from webserver in a form of string and be as much officient as possible.

    • Did you spot any problems regarding async flow in the example above?
    • Is it OK to have .Wait() inside .ContinueWith() in this example
    • Do you see any other problems with this peace of code (keeping aside exception handling for now)?

    Tuesday, November 16, 2010 2:01 AM

Answers

  • There are a few problems with the code as you have it above.  For example, you're disposing of the GetResponseStream synchronously but launching a task to read from it asynchronously; this means it's likely you'll be disposing of the stream before you actually read from it.  As another example, you're only calling BeginRead once, but BeginRead isn't guaranteed to read all of the requested data; you may need to call it multiple times, and the integer returned from EndRead tells you how much data was actually read.

    If I were implementing this (and if I couldn't just wrap WebClient as I demonstrated in a previous post), I would utilize mechanisms like those I outlined in my blog post at: http://blogs.msdn.com/b/pfxteam/archive/2010/11/21/10094564.aspx. In particular, this could be done relatively cleanly using a variant of the Iterate method shown in that post.  Imagine I wrote the following helper function:

    public static void Iterate<TResult>(IEnumerable<Task> asyncIterator, TaskCompletionSource<TResult> tcs) 

        var enumerator = asyncIterator.GetEnumerator(); 
        Action<Task> recursiveBody = null;
        recursiveBody = completedTask =>
        {
            if (completedTask != null && completedTask.IsFaulted)
            {
                tcs.TrySetException(completedTask.Exception.InnerExceptions);
                enumerator.Dispose();
            }
            else if (enumerator.MoveNext())
            {
                enumerator.Current.ContinueWith(recursiveBody, TaskContinuationOptions.ExecuteSynchronously);
            }
            else enumerator.Dispose();
        };
        recursiveBody(null); 
    }

    Then I could use that helper to rewrite your method more like the following:

    public static Task<string> FetchAsync()
    {
        var tcs = new TaskCompletionSource<string>();
        Iterate(FetchAsyncCore(tcs), tcs);
        return tcs.Task;
    }

    private static IEnumerable<Task> FetchAsyncCore(TaskCompletionSource<string> tcs)
    {
        string url = "http://www.snee.com/xml/crud/posttest.cgi";
        var message = Encoding.ASCII.GetBytes("Hello World!");

        var request = (HttpWebRequest)WebRequest.Create(url);
        request.Method = WebRequestMethods.Http.Post;

        // Get the request stream
        var getRequestStream = Task.Factory.FromAsync<Stream>(request.BeginGetRequestStream, request.EndGetRequestStream, null);
        yield return getRequestStream;
        var requestStream = getRequestStream.Result;

        // Write the message to it
        var writeToRequestStream = Task.Factory.FromAsync(requestStream.BeginWrite, requestStream.EndWrite, message, 0, message.Length, null);
        yield return writeToRequestStream;
        requestStream.Close();

        // Get the response
        var getResponse = Task.Factory.FromAsync<WebResponse>(request.BeginGetResponse, request.EndGetResponse, null);
        yield return getResponse;

        // Get the response stream
        using (var response = (HttpWebResponse)getResponse.Result)
        using (var responseStream = response.GetResponseStream())
        {
            // Copy all data from the response stream
            var output = new MemoryStream();
            var buffer = new byte[response.ContentLength > 0 ? response.ContentLength : 0x100];
            while (true)
            {
                var read = Task<int>.Factory.FromAsync(responseStream.BeginRead, responseStream.EndRead, buffer, 0, buffer.Length, null);
                yield return read;
                if (read.Result == 0) break;
                output.Write(buffer, 0, read.Result);
            }

            // Decode the data and store the result
            tcs.TrySetResult(Encoding.ASCII.GetString(output.ToArray()));
        }
    }

    I hope that helps.

    Tuesday, December 07, 2010 2:00 AM
    Owner

All replies

  • Hi Koistya-

    If your goal is to implement FetchAsync that returns a Task<string>, there's an easier way to do this, as the core logic you need is already implemented in WebClient, e.g. something like:

    public static Task<string> FetchAsync()
    {
    const string url = "http://www.snee.com/xml/crud/posttest.cgi", message = "Hello World!" var tcs = new TaskCompletionSource<string>();

    var wc = new WebClient(); wc.UploadtringCompleted += (s,e) => { if (e.Error != null) tcs.SetException(e.Error); else tcs.SetResult(e.Result); }; wc.UploadStringAsync(new Uri(url), "POST", message); return tcs.Task; }

     

    Additionally, if you're interested in playing around with the Async CTP available at http://msdn.com/vstudio/async, this becomes very easy to do using the lower-level support you're trying to use.
    Wednesday, November 17, 2010 9:31 PM
    Owner
  • No, I am looking to implement it with WebRequest and C# 4.0 (without Async CTP)
    http://navin.biz
    Thursday, November 18, 2010 1:17 PM
  • There are a few problems with the code as you have it above.  For example, you're disposing of the GetResponseStream synchronously but launching a task to read from it asynchronously; this means it's likely you'll be disposing of the stream before you actually read from it.  As another example, you're only calling BeginRead once, but BeginRead isn't guaranteed to read all of the requested data; you may need to call it multiple times, and the integer returned from EndRead tells you how much data was actually read.

    If I were implementing this (and if I couldn't just wrap WebClient as I demonstrated in a previous post), I would utilize mechanisms like those I outlined in my blog post at: http://blogs.msdn.com/b/pfxteam/archive/2010/11/21/10094564.aspx. In particular, this could be done relatively cleanly using a variant of the Iterate method shown in that post.  Imagine I wrote the following helper function:

    public static void Iterate<TResult>(IEnumerable<Task> asyncIterator, TaskCompletionSource<TResult> tcs) 

        var enumerator = asyncIterator.GetEnumerator(); 
        Action<Task> recursiveBody = null;
        recursiveBody = completedTask =>
        {
            if (completedTask != null && completedTask.IsFaulted)
            {
                tcs.TrySetException(completedTask.Exception.InnerExceptions);
                enumerator.Dispose();
            }
            else if (enumerator.MoveNext())
            {
                enumerator.Current.ContinueWith(recursiveBody, TaskContinuationOptions.ExecuteSynchronously);
            }
            else enumerator.Dispose();
        };
        recursiveBody(null); 
    }

    Then I could use that helper to rewrite your method more like the following:

    public static Task<string> FetchAsync()
    {
        var tcs = new TaskCompletionSource<string>();
        Iterate(FetchAsyncCore(tcs), tcs);
        return tcs.Task;
    }

    private static IEnumerable<Task> FetchAsyncCore(TaskCompletionSource<string> tcs)
    {
        string url = "http://www.snee.com/xml/crud/posttest.cgi";
        var message = Encoding.ASCII.GetBytes("Hello World!");

        var request = (HttpWebRequest)WebRequest.Create(url);
        request.Method = WebRequestMethods.Http.Post;

        // Get the request stream
        var getRequestStream = Task.Factory.FromAsync<Stream>(request.BeginGetRequestStream, request.EndGetRequestStream, null);
        yield return getRequestStream;
        var requestStream = getRequestStream.Result;

        // Write the message to it
        var writeToRequestStream = Task.Factory.FromAsync(requestStream.BeginWrite, requestStream.EndWrite, message, 0, message.Length, null);
        yield return writeToRequestStream;
        requestStream.Close();

        // Get the response
        var getResponse = Task.Factory.FromAsync<WebResponse>(request.BeginGetResponse, request.EndGetResponse, null);
        yield return getResponse;

        // Get the response stream
        using (var response = (HttpWebResponse)getResponse.Result)
        using (var responseStream = response.GetResponseStream())
        {
            // Copy all data from the response stream
            var output = new MemoryStream();
            var buffer = new byte[response.ContentLength > 0 ? response.ContentLength : 0x100];
            while (true)
            {
                var read = Task<int>.Factory.FromAsync(responseStream.BeginRead, responseStream.EndRead, buffer, 0, buffer.Length, null);
                yield return read;
                if (read.Result == 0) break;
                output.Write(buffer, 0, read.Result);
            }

            // Decode the data and store the result
            tcs.TrySetResult(Encoding.ASCII.GetString(output.ToArray()));
        }
    }

    I hope that helps.

    Tuesday, December 07, 2010 2:00 AM
    Owner
  • Stephen,
    I have a similar issue on which I would like to upload a file without multipart/form-data content-type. It looka like WebClient forces to use multipart/form-data content type, which is a problem if the server requires something else.

    What would be the preferred solution?

    Thanks,
    P.

    Friday, February 04, 2011 9:16 PM
  • Hi MsdnDev-

    WebClient is built on top of HttpWebRequest/HttpWebResponse and encapsulates several common patterns for downloading and uploading data.  If you need to do something outside of the bounds of what WebClient provides, you can build similar functionality yourself on top of HttpWebRequest/HttpWebResponse.  You might be able to take advantage of solutions built by others to handle this kind of thing, e.g. http://stackoverflow.com/questions/219827/multipart-forms-from-c-client.

    I hope that helps.

    Sunday, February 06, 2011 3:25 PM
    Owner