locked
retrieving large data set of record in WCF RRS feed

  • Question

  • User-1141190189 posted

    I am using WCF service to retrieve large amount of data from database (about 20,000) record.

    what is the required configuration in web.config to handle this issue?

    thanks 

    Tuesday, December 18, 2018 4:01 PM

All replies

  • User1120430333 posted

    I am using WCF service to retrieve large amount of data from database (about 20,000) record.

    what is the required configuration in web.config to handle this issue?

    thanks 

     IMO, it would be not to use dataset with datatable, but rather, you can  use a collection DTO(s) as a datacontract to WCF.

    https://en.wikipedia.org/wiki/Data_transfer_object

    https://dzone.com/articles/reasons-move-datatables

    https://www.codeproject.com/Articles/1050468/Data-Transfer-Object-Design-Pattern-in-Csharp

    http://lauteikkehn.blogspot.com/2012/03/datatable-vs-list.html

    http://www.mindscapehq.com/documentation/lightspeed/Building-Distributed-Applications-/Building-WCF-Services-using-Data-Transfer-Objects

    The connectionstring to the database must be in the WCF service config file.

    Tuesday, December 18, 2018 8:27 PM
  • User-330142929 posted

    Hi Sanfoor,

    As far as I know. when the transmitted message is too large, We usually try to configure the following configuration on the client-side and server-side.

    <binding name="mybinding" maxBufferPoolSize="2147483647" maxReceivedMessageSize="2147483647" maxBufferSize="2147483647" transferMode="Streamed" sendTimeout="00:10:00" receiveTimeout="00:10:00">
              <readerQuotas maxDepth="2147483647" maxStringContentLength="2147483647" maxArrayLength="2147483647" maxBytesPerRead="2147483647" />

    Don’t forget to apply the configuration between the client-side and server-side to the service endpoint (you should add this configuration on both the client-side and server-side endpoints).

    <endpoint binding="basicHttpBinding" bindingConfiguration="mybinding" contract="BestService.IService" />


    Feel free to let me know if the problem still exists.

    Best Regards

    Abraham

    Wednesday, December 19, 2018 3:09 AM
  • User-1141190189 posted

    Thank you Abraham

    actually i have tried the above configurations but still i cant retrieve large set of records. the problem is that i need to retrieve about 150000 record at each call.

    i am using sql view and in the c# side iam using SqlDataReader and fill the result in List.

    any advise please?

    Wednesday, December 19, 2018 12:10 PM
  • User-330142929 posted

    Hi Sanfoor,
    I would like you could share your error details with me. Do you set up the above configuration on both the server side and client side? Do you try to retrieve less data so that excludes the other problem?

    Best Regards

    Abraham

    Thursday, December 20, 2018 6:04 AM
  • User1120430333 posted

    Sanfoor

    Thank you Abraham

    actually i have tried the above configurations but still i cant retrieve large set of records. the problem is that i need to retrieve about 150000 record at each call.

    i am using sql view and in the c# side iam using SqlDataReader and fill the result in List.

    any advise please?

    Why do you call a List/collection  a dataset?  It's not an ADO.NET Dataset. Most likely, you can't bring over a 150,000 objects/rows/data based on an undetermined amount of data in each object/row. The default  max amounts in the readerquotas are 65,000 bytes, which you'll have to start changing the default values in the clinet-side config file, to their max values.

    https://www.telerik.com/forums/large-data-not-transfer-using-wcf-service-having-3-layer-application-in-the-wcf

    Thursday, December 20, 2018 10:19 AM