Retrieving large amount of data from WCF service

Question

User-1068299365 posted
Hi All, I am building WCF service that retrieves more than 2,00,000 records from database. Currently I am trying both basicHttpBinding and wsHttpBinding binding but getting following error 'Connection was closed forcefully'. also sometimes gets following error 'An error occurred while receiving the HTTP response to http://localhost:8093/Service.svc. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details' Please reply with the solution.

Answer 1

User388626632 posted

I think this error regarding message size

please refer to the following links

http://stackoverflow.com/questions/10924125/an-error-occurred-while-receiving-the-http-response-error

http://stackoverflow.com/questions/7351676/wcf-maxreceivedmessagesize-error

http://stackoverflow.com/questions/5459697/the-maximum-message-size-quota-for-incoming-messages-65536-has-been-exceeded

http://stackoverflow.com/questions/14134107/how-to-increase-maxreceivedmessagesize-when-calling-a-wcf-from-c-sharp

Answer 2

User-1068299365 posted
I tried various options listed in these links. still doesn't work. While using wsHttpBinding i got "Content Type application/soap+xml; charset=utf-8 was not supported by service http://localhost:8093/Service.svc. The client and service bindings may be mismatched" error. I checked that both service and client uses same wsHttpBinding. Can anyone please give both service and client code that retrieves more than 2,00,000 records from WCF service. I am hosting my service on local IIS.

Answer 3

User-488622176 posted

are you sure the error is in the WCF layer? Did you put a try-cath around your service code to handle exceptions? This kind of errors also occurs when an exception occurs in the wcf service

Answer 4

User-37275327 posted

Available workaround, increase the request lenght, X denotes number.

<system.web>    
<httpRuntime maxRequestLength="XXXXXX" />
</system.web>

Answer 5

User260886948 posted

Hi,

Could you please post your web configure file here to find a good solution?

Best Regards.

Answer 6

User-1068299365 posted
My web.config file of wcf client <configuration> <system.web> <compilation debug="true" targetFramework="4.0"/> </system.web> <system.serviceModel> <bindings> <basicHttpBinding> <binding name="BasicHttpBinding_IService" closeTimeout="00:10:00" openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:10:00" allowCookies="false" bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard" maxBufferSize="2147483647" maxBufferPoolSize="2147483647" maxReceivedMessageSize="2147483647" messageEncoding="Text" textEncoding="utf-8" transferMode="Buffered" useDefaultWebProxy="true"> <readerQuotas maxDepth="32" maxStringContentLength="2147483647" maxArrayLength="2147483647" maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647"/> <security mode="None"> <transport clientCredentialType="None" proxyCredentialType="None" realm=""/> <message clientCredentialType="UserName" algorithmSuite="Default"/> </security> </binding> </basicHttpBinding> </bindings> <client> <endpoint address="http://localhost:8095/Service.svc" binding="basicHttpBinding" bindingConfiguration="BasicHttpBinding_IService" contract="ServiceReference1.IService" name="BasicHttpBinding_IService"/> </client> </system.serviceModel> </configuration> Also on Service side, i write attribute as: [ServiceBehavior (InstanceContextMode=InstanceContextMode.PerSession,ConcurrencyMode=ConcurrencyMode.Multiple)] Pls suggest a good soultion to tackle this problem.

Answer 7

User-1068299365 posted

I am still having the same problem. After analysis, it is found that problem is not in database level but there is problem during transport of large data from WCF service to client.

 Does JSON help me in this situation?

Answer 8

User441938976 posted

Yes, Compare to XML, JSON is very light. Its reduce the response time.

You can define method by below code -

[OperationContract]
    [System.ServiceModel.Web.WebInvoke(Method = "GET", ResponseFormat = System.ServiceModel.Web.WebMessageFormat.Json, BodyStyle = System.ServiceModel.Web.WebMessageBodyStyle.Wrapped, UriTemplate = "json/{id}")]
    string JSONData(string id);

Answer 9

User220959680 posted

JSON help me in this situation

XML is verbose. 

<firstName>umesh</firstName>

JSON can certainly reduce the data being transferred due to its concise format.

firstName: "umesh";

still having the same problem. After analysis, it is found that problem is not in database level but there is problem during transport of large data from WCF service to client.

Database jobs are standard practice to transport large amounts of data. when the service is adopted to transport large amounts of the data make sure that below properties are set in the service and update the client to have relevant configuration. Refer http://msdn.microsoft.com/en-us/library/ms731361.aspx for more info on below configuraiton properties 

OpenTimeout
closeTimeout
receiveTimeout
sendTimeout
maxReceivedMessageSize
maxBufferSize
maxBufferPoolSize

Answer 10

User-488622176 posted

Yes and no. JSON has less overhead then XML, enabling to submit the same data in smaller physical size. It will probably avoid issues with the given length, but I doubt it will be a solution (it will postpone the problem to larger sizes)

Answer 11

User503812343 posted

RESTful services with JSON will help you, as you do not need to mention any size in config and it will reduce XML overhead.

See how to create RESTful Services and return JSON data

http://dotnetmentors.com/wcf/how-to-create-wcf-restful-services.aspx

http://dotnetmentors.com/wcf/wcf-rest-service-to-get-or-post-json-data-and-retrieve-json-data-with-datacontract.aspx