locked
httpclient "a task was canceled! RRS feed

  • Question

  • I've started using the new httpClient in an existing project.

    I have upgraded to use async and get a lot of "a task was canceled" when calling _Client.GetAsync

    I have a single client that gets called 1000's times.

    By adding some debug code I think this may be a time out error (the message needs to be way more helpful!). I can see that the cancellation is early always after 30 seconds.

    The website that it is calling to is very fast and responsive and turns requests around in under 2 seconds even under load.

    What I think might be happening is that the httpclient is queuing up requests and they are timing out before they even hit the server.

    Is this possible?

    If so , how many aysnc calls are made in parallel ?

    Can I control this to say max of 3 async requests


    Can I set  the timeout so it is only from the point at which the request is sent to the server?

    Sunday, May 22, 2016 1:12 PM

Answers

  • Just to update on what I have found having got some more data and looked at the HTTP client code.

    If you look at HttpClient source you can see that the timeout is used for  a cancellation token that applies to the entire async operation.

    If you have a throughput issue or have throttled your max connections the client may be slow to turn around the full async operation.

    The time for the stream operation is ReadWriteTimeout.

    So what I have done is set the client timeout to be much higher and set the ReadWriteTimeout to 30 seconds.

    I also confirmed that my new code now being async and using TPL instead of threading is able to spin up requests much faster. So between this and the timeout "issue" a lot of my connections were being cancelled as the full async operation didn't complete.

    It isn't 100% resolved yet, but I think it is a matter of tuning.

    Monday, May 23, 2016 10:14 PM

All replies

  • How to: Make Multiple Web Requests in Parallel by Using Async and Await (Visual Basic)

    Using the code you have then 20 requests would be 30 seconds on any given day I suppose if run synchronously.

    You can also download WireShark to monitor your connections to see what may be occuring.


    La vida loca

    Sunday, May 22, 2016 1:38 PM
  • If the link provided my MrMonkeyBoy does not solve the issue, you'll need to post the code you are using so that we can see how you've attempted to implement the async functionality.

    Reed Kimble - "When you do things right, people won't be sure you've done anything at all"

    Sunday, May 22, 2016 5:28 PM
  • I'm not seeing a major difference to my code other than the number of request / volume.  These are all endpoints that work with my previous code. 

    I have a class where I inject the _Client so it is reused. The class then makes a few thousand calls using :

    Public Async Function GetPage(AbsoluteUrl As String) As Tasks.Task(Of PageResponse)

    httpResponse = Await _Client.GetAsync(AbsoluteUrl)

    '  code omitted for brevity 

    Dim html As String = Await httpResponse.content.ReadAsStringAsync

    ' parse HTML ot make PageResponse class

    ' pageResponse is just meta data.

    return PageResponse

    end function


    As an experiment I set the timeout for 5 minutes and I got no errors. Each of the connections themselves took < 2 seconds server side as before.

    Previously when it was sync code I only had 3 threads and so there were never more than 3 requests so they all processed as and when fired. 

    What I *think* is going on here is that because im using async I'm handing requests to the httpclient very quickly. The httpclient is stacking up the requests and the timeout time includes the time waiting on the client to send causing them to fail. 

    Is this correct?

    So back to my original questions...how can I set the timeout to be from the point of sending the request to the point of queuing.


    Sunday, May 22, 2016 6:10 PM

  • Previously when it was sync code I only had 3 threads and so there were never more than 3 requests so they all processed as and when fired. 

    What I *think* is going on here is that because im using async I'm handing requests to the httpclient very quickly. The httpclient is stacking up the requests and the timeout time includes the time waiting on the client to send causing them to fail. 

    Is this correct?


    Not because you use async, but because you use it most probably wrong. 
    Did you try the code in the link from mr. Monkeyboy?

    Be aware it makes no sense to call a website thousand times. Http is normal limited to a few connections.

    You can use the tool mr. Monkeyboy provided.


    Success
    Cor

    Sunday, May 22, 2016 7:02 PM
  • Yes, but it only happens when there are a large number of calls.

    At around  a few hundred calls I start to see errors.

    If there are a few thousand I get a lot more errors.


    • If I reuse the client and there are 1000 requests I'm assuming it queues these up is this correct?
    • Is it the servicepoint max connections that determines the maximum number of connections?
    • If the item is in the queue does the timeout timer start when it is added to the queue?
    • Is there a mechanism to timeout the actual read time,
      perhaps : webRequestHandler.ReadWriteTimeout?

    Sunday, May 22, 2016 7:32 PM
  • You can also see this link Asynchronous Programming with Async and Await (Visual Basic).

    Note that the image under "What Happens in an Async Method" is really helpful in C# huh?


    La vida loca

    Sunday, May 22, 2016 8:38 PM
  • I've seen that flow diagram before, it's very handy indeed and that is more or less how my application is working. Previously it had to wait on the remainder of the thread, but now it can just create more requests.

    I've also discovered in .Net 4.5 that the max number of service point connections default is better handled, so I have commented out where I explicitly set it.

    I've shipped a tweak in to production that sets the client timeout to 10 mins and the read/write to 30 seconds. With the added instrumentation hopefully it will show what is really going on.

    The behavior of the HttpClient seems a bit obscure unless I'm missing some documentation. Perhaps a flow diagram of how it queues and processes many requests async and where the various timeouts are applicable.

    Sunday, May 22, 2016 10:11 PM
  • I've seen that flow diagram before, it's very handy indeed and that is more or less how my application is working. Previously it had to wait on the remainder of the thread, but now it can just create more requests.

    I've also discovered in .Net 4.5 that the max number of service point connections default is better handled, so I have commented out where I explicitly set it.

    I've shipped a tweak in to production that sets the client timeout to 10 mins and the read/write to 30 seconds. With the added instrumentation hopefully it will show what is really going on.

    The behavior of the HttpClient seems a bit obscure unless I'm missing some documentation. Perhaps a flow diagram of how it queues and processes many requests async and where the various timeouts are applicable.

    I read in a thread about phone apps I believe that only 128 could be launched with this method with a max of like 135 or something but I figured that was due to phones limitations so I didn't provide that link. Nor could I find anything relevant to PC apps with this limitation for HTTPClient. I'll try to find that thread again and if I do I'll place the link in this post but I figure it isn't relevant.

    Well I can't find that particular thread but possibly these threads regarding a server being connected to allowing only x amount of connections at a time may help.

    Is async HttpClient from .Net 4.5 a bad choice for intensive load applications?

    .NET HTTPClient Asynchronous Limitations

    Async/Await maximum number of concurrent http requests

    Don't Block on Async Code


    La vida loca

    Sunday, May 22, 2016 11:30 PM
  • Just to update on what I have found having got some more data and looked at the HTTP client code.

    If you look at HttpClient source you can see that the timeout is used for  a cancellation token that applies to the entire async operation.

    If you have a throughput issue or have throttled your max connections the client may be slow to turn around the full async operation.

    The time for the stream operation is ReadWriteTimeout.

    So what I have done is set the client timeout to be much higher and set the ReadWriteTimeout to 30 seconds.

    I also confirmed that my new code now being async and using TPL instead of threading is able to spin up requests much faster. So between this and the timeout "issue" a lot of my connections were being cancelled as the full async operation didn't complete.

    It isn't 100% resolved yet, but I think it is a matter of tuning.

    Monday, May 23, 2016 10:14 PM
  • Hi DanBayley,

    >>” I also confirmed that my new code now being async and using TPL instead of threading is able to spin up requests much faster. So between this and the timeout "issue" a lot of my connections were being cancelled as the full async operation didn't complete.

    I’m glad to know you have found a solution that could achieve your goal and thank you for sharing the solution.

    Besides, you could mark your solution as answer to close this thread so that to help someone who have the same issues. Thank you for your understanding and support.

    Best Regards,

    Albert Zhang

    Friday, June 3, 2016 5:46 AM