serialized Object size is greater than MaxBufferSize on server

Question

Oke, i finally got the Cache cluster and host working after i stupid mistake on my part (forgot to enable the port forwarding with the enable/disable checkboxes). Connecting to the DataCacheFacotry goes all smooth. So i tried to add a simple string to the cache, cache.Add("test", "test");, doesn't work when connecting from a client wich is not on the same machine as the cluster is. The server replies to me with an error code "<ERRCA0016>:SubStatus<ES0001> serialized Object size is greater than MaxBufferSize on server". I googled for over 2 hours now and tried a lot of stuff. Tried to add the maxbuffersize etc. to my cluster configuration but it doesnt seem to have any effect. Microsoft, anybody, give some insight on how to solve this error.

Answer 1

Oke, i finally got the Cache cluster and host working after i stupid mistake on my part (forgot to enable the port forwarding with the enable/disable checkboxes). Connecting to the DataCacheFacotry goes all smooth. So i tried to add a simple string to the cache, cache.Add("test", "test");, doesn't work when connecting from a client wich is not on the same machine as the cluster is. The server replies to me with an error code "<ERRCA0016>:SubStatus<ES0001> serialized Object size is greater than MaxBufferSize on server". I googled for over 2 hours now and tried a lot of stuff. Tried to add the maxbuffersize etc. to my cluster configuration but it doesnt seem to have any effect. Microsoft, anybody, give some insight on how to solve this error.

Anybody? Why is this so badly documentated by microsoft. The errormessages doesn't provide anything usefull or guidance to solve the problem. This is unacceptable, how can we use a product what is so poorly documentated.

Answer 2

Does this solve your issue: http://social.msdn.microsoft.com/Forums/en-US/velocity/thread/3e4a3b81-717e-4573-acef-37cc43beee66. For client's max buffer size you can set it programmatically also in TransportProperties under DataCacheFactory Configuration.

Answer 3

I have the same problem and i am just trying to put or get a small string
This is really bad!!!

 

http://social.msdn.microsoft.com/Forums/en-US/velocity/thread/301f200c-6372-44ea-87c4-da649bc2e83c

Answer 4

Why would that solve the problem. he says that he is just putting a small string.

Connecting to the DataCacheFacotry goes all smooth. So i tried to add a simple string to the cache, cache.Add("test", "test");

Answer 5

Wee need answers!!

Answer 6

When looking at the Intellitrace in Visual Studio 2010 you can find out the real exceptions instead of
the
ErrorCode<ERRCA0016>:SubStatus<ES0001>:The connection was terminated, possibly due to server or network problems or serialized Object size is greater than MaxBufferSize on server. Result of the request is unknown.

I found out that the server is complaining about no DNS entries exists for ptpseelm-nt4097.
Then i started to add the servername and Ip in the local hosts file but still it dosent work.
im getting really tired of theese strange behaviours.

Answer 7

And its working within the domain just not between DOMAIN 1 and DOMAIN 2

Answer 8

I have the same problem. I'am trying to put a DataSet from SQL in the cache. I make a TOP 10000 and it crashes. I make a TOP 1 and it passes. So it's really a problem of MaxBufferSize.

If I use configuration.TransportProperties.MaxBufferSize to modify it it continues to  crash.

I need help!

 

Thanks

Answer 9

Hello. Can I clarify a few things? You're saying that when your DataSet contains 10000 records that you crash with the exception in this thread? You've changed the maxbuffersize but still get the exception? It's possible that the serialized size of your dataset is bigger than you expect. Can you set the MaxBufferSize to something much larger than you think you need to handle the 10,000 rows?

Also, for others reading the earlier thread on this, it may have had to do with the cross-domain issues. See http://social.msdn.microsoft.com/Forums/en-US/velocity/thread/513f0c1f-fab3-4122-86a6-6814fec6ff5a at the end of the thread to see that discussion.

Thanks.

Jason Roth

Answer 10

Hi all,

 

I'm getting the same error.

 

I'm using the same domain. I tested it on the server machine when client running under test.exe and under IIS process w3wp.

Looks like the issue related to the size of stored data.

 

I have the problem when trying to store XML created from 10000 DB records. If the the XML is smaller (from 1000 records) Put( ) succeeded.

There is a code snippet:

DataCacheTransportProperties transport = new DataCacheTransportProperties();

transport.MaxBufferSize = int.MaxValue; - doesn't solve the problem

string ret = RetrieveFromDBAsXml("", 1000);

_CacheTest.Put("key", ret);  - succeeded

ret = RetrieveFromDBAsXml("", 10000);

_CacheTest.Put("key2", ret); - failed

Any ideas?

Thanks in advance,

Evgeny

 

 

 

Answer 11

More details related to error ErrorCode<ERRCA0016>:SubStatus<ES0001>.

Retrieved XML is about 20MB.

In config file I put value about 100MB - still failing.

 

    <transportProperties connectionBufferSize="131072" maxBufferPoolSize="268435456"

                           maxBufferSize="99709942" maxOutputDelay="2" channelInitializationTimeout="60000"

                           receiveTimeout="600000"/>

failing as well:

<transportProperties connectionBufferSize="99709942" maxBufferPoolSize="99709942"

                           maxBufferSize="99709942" maxOutputDelay="2" channelInitializationTimeout="60000"

                           receiveTimeout="600000"/>

Thanks,

Evgeny

 

Answer 12

Sorry for the extreme delay here. The problem may be that after you return the XML, it has to then be serialized as an object to the cache, which means that it will be even larger than the size of the data in the code (in this case the length of the string).

Even so, it seems that your 100MB should cover it. I have a guess here. I assume you're making this change in the web.config. But there is a similar setting in the cache cluster configuration settings that you can make (when you call Export-CacheClusterConfig and Import-CacheClusterConfig). If you make this change under the advanced properties node, it may solve the problem by changing this limit on both the client and the server.

Jason Roth

Answer 13

i got the same issue which u all met.

anybody found the solution for this, if  yes then please share here.

---

Thanks In Advance, Jeyaseelan

Answer 14

1) Please check Total data size on server (cached objects) by using performance counter

2) set maxBufferSize and maxBufferPoolSize setting at client side and server(Server Cluster configuration) side. Max buffer size and max buffer pool size to 2147483647 bytes(2047 MB) (max limit)

3) set maxConnectionsToServer =3

4) Trace and Log Errors please review: http://msdn.microsoft.com/en-us/library/ff921010.aspx

5) check for Available size for caching under Cluster Cache Host Config.

6)Hopefully serialization and compression will solve your problem. Good luck First try with XML binary serialization, If it will not work then go with commented plain binary serialization

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.ApplicationServer.Caching;
using System.Runtime.Serialization.Formatters.Binary;
using System.IO;
using System.IO.Compression;
using System.Xml;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Formatters;
using System.Configuration;


namespace SiteComponents
{

    public class ConfigCache
    {

        public static byte[] SerializeXmlBinary(object obj)
        {
            using (MemoryStream ms = new MemoryStream())
            {
                using (XmlDictionaryWriter wtr = XmlDictionaryWriter.CreateBinaryWriter(ms))
                {
                    NetDataContractSerializer serializer; serializer = new NetDataContractSerializer();
                    serializer.WriteObject(wtr, obj);
                    ms.Flush();
                }
                return ms.ToArray();
            }
        }
        public static object DeSerializeXmlBinary(byte[] bytes)
        {
            using (XmlDictionaryReader rdr = XmlDictionaryReader.CreateBinaryReader(bytes, XmlDictionaryReaderQuotas.Max))
            {
                NetDataContractSerializer serializer; serializer = new NetDataContractSerializer();
                serializer.AssemblyFormat = FormatterAssemblyStyle.Simple;
                return serializer.ReadObject(rdr);
            }
        }


        public static byte[] CompressData(object obj)
        {
            byte[] inb = SerializeXmlBinary(obj);
            byte[] outb;
            using (MemoryStream ostream = new MemoryStream())
            {
                using (DeflateStream df = new DeflateStream(ostream, CompressionMode.Compress, true))
                {
                    df.Write(inb, 0, inb.Length);
                } outb = ostream.ToArray();
            } return outb;
        }

        public static object DecompressData(byte[] inb)
        {
            byte[] outb;
            using (MemoryStream istream = new MemoryStream(inb))
            {
                using (MemoryStream ostream = new MemoryStream())
                {
                    using (System.IO.Compression.DeflateStream sr =
                        new System.IO.Compression.DeflateStream(istream, System.IO.Compression.CompressionMode.Decompress))
                    {
                        sr.CopyTo(ostream);
                    } outb = ostream.ToArray();
                }
            } return DeSerializeXmlBinary(outb);
        }

        //public static byte[] SerializeBinary(object obj)
        //{
        //    using (MemoryStream ms = new MemoryStream())
        //    {
        //        BinaryFormatter serializer; serializer = new BinaryFormatter();
        //        ms.Position = 0;

        //        serializer.Serialize(ms, obj);
        //        ms.Flush();
        //        return ms.ToArray();
        //    }
        //}
        //public static object DeSerializeBinary(MemoryStream ms)
        //{
        //    //using (MemoryStream ms = new MemoryStream())
        //    //{
        //    BinaryFormatter serializer; serializer = new BinaryFormatter();
        //    ms.Position = 0;
        //    object obj = serializer.Deserialize(ms);
        //    return obj;
        //    //}
        //}


        //public static byte[] CompressData(object obj)
        //{
        //    byte[] inb = SerializeBinary(obj);
        //    byte[] outb;
        //    using (MemoryStream ostream = new MemoryStream())
        //    {
        //        using (DeflateStream df = new DeflateStream(ostream, CompressionMode.Compress, true))
        //        {
        //            df.Write(inb, 0, inb.Length);
        //        } outb = ostream.ToArray();
        //    } return outb;
        //}

        //public static object DecompressData(byte[] inb)
        //{
        //    byte[] outb;
        //    using (MemoryStream istream = new MemoryStream(inb))
        //    {
        //        using (MemoryStream ostream = new MemoryStream())
        //        {
        //            using (System.IO.Compression.DeflateStream sr =
        //                new System.IO.Compression.DeflateStream(istream, System.IO.Compression.CompressionMode.Decompress))
        //            {
        //                sr.CopyTo(ostream);
        //            } outb = ostream.ToArray();
        //            return DeSerializeBinary(ostream);
        //        }
        //    }
        //}
        private static DataCacheFactory _cacheFactory;
        private static object _cacheFactoryLock = new object();
        private static DataCache _operationCache;
        private static object _operationCacheLock = new object();

        public static bool IsExist(string CacheKey)
        {
            if (OperationCache.Get(CacheKey) != null)
                return true;
            else return false;
        }

        public static Object Get(string CacheKey)
        {
            return DecompressData((byte[])OperationCache.Get(CacheKey));
        }

        public static Object AddAndUpdate(string CacheKey, Object obj, int timeSpan)
        {
            if (obj == null)
                return null;
            OperationCache.Put(CacheKey, CompressData(obj));
            OperationCache.ResetObjectTimeout(CacheKey, TimeSpan.FromMinutes(timeSpan));
            return obj;
        }

        public static bool Remove(string CacheKey)
        {
            if (OperationCache.Get(CacheKey) != null)
            {
                OperationCache.Remove(CacheKey);
                return true;
            }
            return false;
        }

        
        private static DataCacheFactory CacheFactory
        {
            
            get
            {
                if (_cacheFactory == null)
                {
                    lock (_cacheFactoryLock)
                    {
                        if (_cacheFactory == null)
                        {
                            try
                            {
                                _cacheFactory = new DataCacheFactory();
                            }
                            catch (Exception ex)
                            {
                                // Perform any logging and rethrow.
                                throw ex;
                            }
                        }
                    }
                }
                return _cacheFactory;
            }
        }

        private static DataCache OperationCache
        {
            get
            {
                if (CacheFactory != null)
                {
                    if (_operationCache == null)
                    {
                        lock (_operationCacheLock)
                        {
                            if (_operationCache == null)
                            {
                                try
                                {
                                    _operationCache = CacheFactory.GetCache("CacheName");
                                }
                                catch (Exception ex)
                                {
                                    // Perform any logging and rethrow.
                                    throw ex;
                                }
                            }
                        }
                    }
                    return _operationCache;
                }
                return null;
            }
        }

    }
}