none
Several 16MB buffers that stay allocated in HttpChannelListener's bufferedWriterPool member (for a total of 218 MB) RRS feed

  • Question

  • Hello,

    I was memory profiling a WCF based service that takes up too much memory, and saw that 218 MB of memory is being held by HttpChannelListeners's bufferedWriterPool, in several entries each taking up 16MB.

    We are configuring the Http binding using the following code:

            private static BasicHttpBinding GetDefaultBinding(ServiceTransport transport, ServiceAuthentication authentication)
            {
                var binding = new BasicHttpBinding();
                binding.Name = "EasyFormNxGBinding";
                binding.HostNameComparisonMode = HostNameComparisonMode.StrongWildcard;
                binding.CloseTimeout = new TimeSpan(0, 1, 0);
                binding.OpenTimeout = new TimeSpan(0, 1, 0);
                binding.ReceiveTimeout = new TimeSpan(0, 10, 0);
                binding.SendTimeout = new TimeSpan(0, 1, 0);
                binding.AllowCookies = false;
                binding.BypassProxyOnLocal = false;
                // settings for the System.ServiceModel.Channels.BufferManager, see explanation http://obsessivelycurious.blogspot.be/2008/04/wcf-memory-buffer-management.html
                // note: very large requests will be handled by allocating a buffer from the heap and not from the buffer pool (this memory will be freed in a garbage collection run) 
                // in buffered transfer mode, the MaxBufferSize MUST be equal to the MaxReceivedMessageSize
                binding.MaxBufferSize = 2147483647;     // 2 GB  (max size of a request that is handled by the buffer, larger requests will be allocated from the heap)
                binding.MaxBufferPoolSize = 10485760;   // 10 MB (total size of the pool buffer --> larger requests will not be handled by the bufferpool)
                binding.MaxReceivedMessageSize = 2147483647;
                binding.MessageEncoding = WSMessageEncoding.Text;
                binding.TextEncoding = Encoding.UTF8;
                binding.TransferMode = TransferMode.Buffered;
                binding.UseDefaultWebProxy = true;
                binding.ReaderQuotas.MaxDepth = 2147483647;
                binding.ReaderQuotas.MaxStringContentLength = 2147483647;
                binding.ReaderQuotas.MaxArrayLength = 2147483647;
                binding.ReaderQuotas.MaxBytesPerRead = 2147483647;
                binding.ReaderQuotas.MaxNameTableCharCount = 2147483647;
    ...
    }

    Should the code line "binding.MaxBufferPoolSize = 10485760;" not limit the maximum pool buffer size to 10 MB and allocate larger buffers from the .NET heap?

    Note: I know that setting several parameters to 2 GB isn't best practice, but these WCF services are hosted in intranet environments and not internet.

    Monday, November 26, 2018 10:52 AM

All replies

  • Hi PDHCoder,

    In my opinion, these data are going to be written to the buffer pool, not cached data. Because it has not yet come out of the channel. And the parameter metric should point to the buffer manager instance class.
    Try to refer to the following official explanation. Wish it is useful to you.
    https://docs.microsoft.com/en-us/dotnet/api/system.servicemodel.basichttpbinding.maxbufferpoolsize?view=netframework-4.0
    https://docs.microsoft.com/en-us/dotnet/api/system.servicemodel.channels.buffermanager?view=netframework-4.0
    Best Regards
    Abraham

    Tuesday, November 27, 2018 6:34 AM
    Moderator
  • Hi Abraham,

    I am asking these questions because our 32bit WCFservice is taking up a lot of memory (typically averages around 600 MB), a big part of which can be explained by WCF related buffers. 218 MB of buffers in WCF is 1/5 of the total memory available to a 32 bit process. As a result, we occasionally get OutOfMemoryExceptions when memory usage rises above 1GB.

    If you say the allocations are there because the data has not yet come out of the channel, and binding.SendTimeout  is set to one minute, then these buffers should be freed after one minute, regardless whether the client receives the response? And these output buffers should not remain allocated in the buffer pool, because the maximum was set to 10 MB (cfr. binding.MaxBufferPoolSize = 10485760).

    Am I correct that the parameter MaxBufferSize does NOT affect the amount of memory that could be allocated permanently in the buffer pool?

    Best regards,

    Paul


    • Edited by PDHCoder Tuesday, November 27, 2018 9:18 AM
    Tuesday, November 27, 2018 9:17 AM
  • Hi PdhCoder,

    >>Am I correct that the parameter MaxBufferSize does NOT affect the amount of memory that could be allocated permanently in the buffer pool?

    Yes. In my opinion, it doesn't make sense to set the Maxbuffersize to be larger than the MaxbufferpoolSize. Because if we can't find a buffer pool of sufficient size to allocate to that buffer, the system allocates the appropriate size of memory to that buffer. This cache area will no longer be returned to the buffer pool, but will be managed by the system garbage collection mechanism,and it will takes up lots of resource.

    Best Regards

    Abraham

    Wednesday, December 5, 2018 6:49 AM
    Moderator