locked
Wcf Data Services Client Large Dataset Performance RRS feed

  • Question

  • Hi,

    We are using Wcf Data Services 5.6 in Wpf client connected to Web Api OData Server. The Reason why we use Web Api OData instead of standard Wcf DataService is because we need to do a more complex projection returning a custom type, which Wcf DataService does not support.

    Use case:

    Load about 100k Records (each record 20 fields), which is, according to fiddler, 33Mb of data. 

    The Format is Json 

    this.Container.Format.UseJson();

    although it does not seem to make much difference compared to Atom.

    One complete call takes about 24 seconds. This is how the time is split between the different layers:

    • Database query - 3 seconds
    • Server Response time - 7 seconds (measured in Fiddler), which means around 4 seconds is spent on serializing the records to Json response.
    • That leaves us with around 17 seconds spent on client side to deserialize the response to poco's

    The question is how can we optimize any of this, but mainly the client side deserialization seems to be very slow.

    Is the client side using Newtonsoft Json.net to deserialize the reponse. If not can the serializer be changed?

    Any ideas appreciated.

    Thanks,
    Stevo





    • Edited by Stevo Zilik Wednesday, October 9, 2013 10:56 AM
    Wednesday, October 9, 2013 10:53 AM

Answers

  • Hi Stevo,

    It sounds like it could be due to the fact that Microsoft's current implementation in WCF DS Client is doing an in memory stream copy of the response from the server before deserialization/materialization starts.

    I have earlier noticed issues with this (particular noticeable on larger payloads) and have reported it here: Performance/throughput issue in WCF DS Client System.Data.Services.Client.QueryResult.ExecuteQuery

    If you could try and run you client code under a profiler you should be able to pinpoint if this is the same issue.

    As far as I understand Microsoft has no plans for fixing this in the near future.

    Again if this is the same issue you could consider rewriting you client code to use the ODataLib ODataMessageReader directly as it doesn't do the stream copy stuff.

    Regards

    Uffe

    Thursday, October 17, 2013 5:21 PM
  • >>And how does that fit with Wcf Data Services Client?

    For this, this article written by hanselman shows how to deal with it.

    Regards.


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Tuesday, October 15, 2013 3:42 AM
    Moderator

All replies

  • Hello,

    >>Is the client side using Newtonsoft Json.net to deserialize the reponse?

    If we use code like below, we are using Json.net.

    SightingType deserializedSightingType = JsonConvert.DeserializeObject<SightingType>(data);

    Regards.


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Thursday, October 10, 2013 8:37 AM
    Moderator
  • Hello,

    >>Is the client side using Newtonsoft Json.net to deserialize the reponse?

    If we use code like below, we are using Json.net.

    SightingType deserializedSightingType = JsonConvert.DeserializeObject<SightingType>(data);


    Hi,

    And how does that fit with Wcf Data Services Client?

    Monday, October 14, 2013 2:40 PM
  • >>And how does that fit with Wcf Data Services Client?

    For this, this article written by hanselman shows how to deal with it.

    Regards.


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Tuesday, October 15, 2013 3:42 AM
    Moderator
  • Hi Stevo,

    It sounds like it could be due to the fact that Microsoft's current implementation in WCF DS Client is doing an in memory stream copy of the response from the server before deserialization/materialization starts.

    I have earlier noticed issues with this (particular noticeable on larger payloads) and have reported it here: Performance/throughput issue in WCF DS Client System.Data.Services.Client.QueryResult.ExecuteQuery

    If you could try and run you client code under a profiler you should be able to pinpoint if this is the same issue.

    As far as I understand Microsoft has no plans for fixing this in the near future.

    Again if this is the same issue you could consider rewriting you client code to use the ODataLib ODataMessageReader directly as it doesn't do the stream copy stuff.

    Regards

    Uffe

    Thursday, October 17, 2013 5:21 PM