none
Configuring BlobEndpoint and CloudBlobClient for maximum update throughput

    Question

  • I'm updating a large number of CloudBlockBlob using a CloudBlobClient per request (it's always the same blob).

    The parameters for the the BlobEndpoint are set as follows:

    blobEndpoint.UseNagleAlgorithm<-false
    blobEndpoint.ConnectionLimit<-1000
    //blobEndpoint.MaxIdleTime<-Timeout.Infinite
    

    What I'm noticing is that after a while, the number of updates per second decreases to a very low rate and I suppose it's due to the number of available connections.

    Do you recommend using the same blob client for multiple requests with a certain level of parallelism? Maybe a different configuration of the endpoint parameters?

    Thanks


    • Edited by Horia Toma Monday, February 27, 2017 6:14 PM
    Monday, February 27, 2017 4:52 PM

All replies

  • Looks like your same question has been answered on Stackoverflow. Also, could you try testing it out with a single blob client to see if it helps improve the performance.
    Tuesday, February 28, 2017 9:15 AM
  • Hi Shibab, 

    The question was not answered yet, I received a suggestion to use the Movement library.

    I will try using a single client and see how it goes, though the parallelism level equivalent to the number of cores is not enough, but I will confirm this later.

    Tuesday, February 28, 2017 9:27 AM
  • I experiment the same kind of behavior when having a single client used by all threads.

    I still think the problem is in the blob client and that connections are not recycled, thus I end with a very low number of active connections.

    Tuesday, February 28, 2017 12:51 PM
  • The solution I found was to set `maxconnection` to 1000 in system.net.

    Worked so much better than the `ConnectionLimit` on the blob endpoint.
    Tuesday, February 28, 2017 3:56 PM