locked
Performance/throughput issue in WCF DS Client System.Data.Services.Client.QueryResult.ExecuteQuery RRS feed

  • Question

  • Hi,

    Consider the following scenario. From client code written with WCF DS Client you are querying an OData server that returns a (large) feed in streamed mode. The first entry in the feed arrives at the client almost immediatly and the last entry arrives after lets say 5 seconds. With client code resembling the following i would expect the first Console.Write almost instantaneously, but it does not - it actually waits untill all data has been retrieved from the server (the 5 seconds).

    var q = from f in context.Files
        select f;
    
    foreach (var f in q)
        Console.Write('.');
    

    Digging a bit into the problem I found that this delay is due to WCF DS Client doing a stream copy of the response stream from the server as part of execute (before enumerating starts or triggered by starting enumerating).

    IlSpy revealse the following implementation of QueryResult.ExecuteQuery (that is called from DataServiceRequest.Execute).

    internal void ExecuteQuery()
    {
    	try
    	{
    		if (this.requestContentStream != null && this.requestContentStream.Stream != null)
    		{
    			this.Request.SetRequestStream(this.requestContentStream);
    		}
    		IODataResponseMessage syncronousResponse = this.RequestInfo.GetSyncronousResponse(this.Request, true);
    		this.SetHttpWebResponse(Util.NullCheck<IODataResponseMessage>(syncronousResponse, InternalError.InvalidGetResponse));
    		if (HttpStatusCode.NoContent != this.StatusCode)
    		{
    			using (Stream stream = this.responseMessage.GetStream())
    			{
    				if (stream != null)
    				{
    					Stream asyncResponseStreamCopy = this.GetAsyncResponseStreamCopy();
    					this.outputResponseStream = asyncResponseStreamCopy;
    					byte[] asyncResponseStreamCopyBuffer = this.GetAsyncResponseStreamCopyBuffer();
    					long num = WebUtil.CopyStream(stream, asyncResponseStreamCopy, ref asyncResponseStreamCopyBuffer);
    					if (this.responseStreamOwner)
    					{
    						if (0L == num)
    						{
    							this.outputResponseStream = null;
    						}
    						else
    						{
    							if (asyncResponseStreamCopy.Position < asyncResponseStreamCopy.Length)
    							{
    								((MemoryStream)asyncResponseStreamCopy).SetLength(asyncResponseStreamCopy.Position);
    							}
    						}
    					}
    					this.PutAsyncResponseStreamCopyBuffer(asyncResponseStreamCopyBuffer);
    				}
    			}
    		}
    	}
    	catch (Exception e)
    	{
    		base.HandleFailure(e);
    		throw;
    	}
    	finally
    	{
    		base.SetCompleted();
    		this.CompletedRequest();
    	}
    	if (base.Failure != null)
    	{
    		throw base.Failure;
    	}
    }
    

    Is there a good reason for the implementation doing so? Or is it a bug? Please explain.

    As I see it there are several problems with doing this.

    1. As described it will take longer to get all data processed on the client.

    2. In scenarios where the client don't want any change tracking but just want the data and do some calculations or handing it off to some other party you get an unnecessary memory consumption hit by keeping the full response in memory.

    I do know I could write the same code using ODataLib and an ODataReader and get a proper streaming model without this overhead, but it is a lot more complex four our partners to go the ODataLib route.

    Regards

    Uffe

    Monday, July 22, 2013 7:57 AM

Answers

  • Hey Uffe,

    We have opened a bug on this performance issue.

    Thanks,

    Chris Robinson - WCF Data Services Team


    This posting is provided "AS IS" with no warranties, and confers no rights.

    Tuesday, July 23, 2013 3:55 PM
    Moderator

All replies

  • Hi,

    The reason is history (sort of). When the WCF DS client was written (.NET 3.5 timeframe) the .NET framework didn't have an XML parser which could read data from an asynchronous stream in a streaming fashion. The XML parser simply consumed a Stream and read from it until the end. There's no reliable implementation using such parser which would let you NOT buffer the entire response when using synchronous APIs to read from the network (remember that the XML parser uses synchronous APIs to read from the stream it is given).

    Since then only in .NET 4.5 a new XML parser which is able to read from asynchronous streams was added. WCF DS Client simply didn't start using it yet. If nothing else this parser is not available on all platforms where the WCF DS Client is supported yet.

    Note that a similar problem applies to ODataLib. By the time it was first created, it was written for .NET 4.0. So no "await" and such. As such it does support asynchronous reading, but it's in fact cheating as it will cache the entire response internally as well (same problem with the XML parser as above). ODataLib would be a bit easier to "fix", but it's still a huge change. Not counting the problem that the async XML parser is not supported on all the platforms where ODataLib is.

    If you're using JSON then obviously the reason for XML doesn't apply. In this case the ODataLib and WCF DS Client were simply not yet implemented that way (given that XML would not work anyway). So they will exhibit the same behavior even in JSON just because the same code runs for both formats.

    Possible workarounds:

    Split the response into multiple responses. If the response size is really big (several MBs) it's usually not a good idea to use that for multiple reasons:

    - It means the server has to be willing to provide such big response, which consumes lot of resources to produce. But that might open the server to easier DoS attacks.

    - If there are proxies/caches along the way between the client and the server, some of them might not like big payloads like this.

    - In case of a network error, the entire payload is treated as faulted. So the client would have to retry the entire big request again.

    The splitting can be done either on the server by using server driven paging which also protects the server from attacks. Client then simply loops until it gets all the data, it just sends more requests this way.

    Or it can be done on the client via $top/$skip.

    The downside of this workaround is that the network latency impacts the overall performance more as it is "hit" more than once.

    Thanks,


    Vitek Karas [MSFT]

    Monday, July 22, 2013 4:20 PM
    Moderator
  • Thanks for the response Vitek.

    Your comment of ODataLib exibiting the same behaviour is not true. As stated if i write the same query using ODataLib directly and perform the reading with an ODataMessageReader with the following code, the reading of the first entity is close to immediatly.

    ODataMessageReaderSettings settings = new ODataMessageReaderSettings();
    settings.MessageQuotas.MaxReceivedMessageSize = Int32.MaxValue;
    settings.CheckCharacters = false;
    settings.DisablePrimitiveTypeConversion = true;
    settings.UndeclaredPropertyBehaviorKinds = ODataUndeclaredPropertyBehaviorKinds.IgnoreUndeclaredValueProperty;
    
    while (nextLink != null)
    {
        HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(nextLink);
        request.Credentials = System.Net.CredentialCache.DefaultCredentials;
        request.Accept = "application/json";
        ClientHttpResponseMessage responseMessage = new ClientHttpResponseMessage((HttpWebResponse)request.GetResponse());
    
        using (ODataMessageReader messageReader = new ODataMessageReader(responseMessage, settings, model))
        {
            ODataReader reader = messageReader.CreateODataFeedReader();
            while (reader.Read())
            {
                if (reader.State == ODataReaderState.EntryEnd)
                {
                    c++;
                    Console.Write('.');
                }
                if (reader.State == ODataReaderState.FeedEnd)
                {
                    ODataFeed feed = (ODataFeed)reader.Item;
                    nextLink = feed.NextPageLink;
                }
            }
        }
    }
    Console.WriteLine();
    Console.WriteLine(string.Concat("MessageReader Completed in: ", DateTime.Now - started));

    Also, I am using server driven paging in my Custom WCF DS Provider. The issue has nothing to do with the pagesize it is however more noticably with larger page sizes. So it will not help with a smaller page size if you take into account all the requests needed to provide all the data the client wan't.

    Actually I found the issue bacause I am using server driven paging. By looking at a Fiddler trace I noticed gaps between the page requests and wondered what caused those gaps. 

    Also please notice I am not talking about async reading, I am simply reading as fast as I can from the mainthread. I do not think that an async model would behave any diferent.

    So the question remains why can ODataLib do it but not the build in ExecuteQuery not?

    Regards

    Uffe


    • Edited by Uffe Lauesen Monday, July 22, 2013 4:58 PM Fixed code to closer match the original example.
    Monday, July 22, 2013 4:57 PM
  • To illustrate my claims i have prepared fiddler timeline traces of the two scenarios.

    The build in ExecuteQuery will show the following time line:

    ExecuteQuery

    The same Query but read with ODataLib ODataMessageReader:

    Please notice that the two Graphs dont have the same axis format.

    As you can se ODataMessageReader completes alot faster and there are no gaps inbetween the requests. You may also know my server driven paging strategy is not with a constant page size, but increaeses by a factor for each page. This strategy is actually very efficient when the client actually want all the data (think BI/Excel PowerPivot scenarios). My service is hosted in a corporate Network I am not that volnurable to hostile DOS attacks. From the Graphs you may also notice that the server really is streaming the contents and time to first byte is quite fast. The server is not using any more resouces/memory for providing lager page sizes.

    So if the Stream copying in ExecuteQuery is by design I will recommend everybody to have a closer look at the ODataLib ODataMessageReader if performance and overall througput is of concern.

    Regards

    Uffe


    • Edited by Uffe Lauesen Monday, July 22, 2013 5:20 PM Clarified axis format.
    Monday, July 22, 2013 5:19 PM
  • Sorry I wasn't too clear. My comment was about async reading (so ODataMessageReader.CreateODataFeedReaderAsync and then ReadAsync and so on), in that case even ODataLib will buffer the entire response. If you use synchronous methods then yes, ODataLib doesn't buffer anything.

    I'm not 100% sure why WCF DS in the synchronous code path caches the entire response... I will leave that up to somebody with more knowledge of the client code. (My guess would be that it simply implements a similar mechanism in both sync and async code paths, since that way it's simpler).

    Note on the server security: If you're running this on internal network then I agree this is mostly a moot point. But note that even fully streaming server (time to first byte being fast) doesn't prevent problems with large responses, since the attacker will not close the connection, it will happily read all of it causing the server to process everything in the response anyway (and how fast it got written to the wire doesn't matter much). This is not necessarily about memory pressure (since streaming mostly fixes that), but overall server load. With very large responses attacker can degrade server performance with fewer requests since they will take longer to process (and consume CPU/IO/thread resources). But as you mentioned above... on internal network this is moot :-)

    Thanks,


    Vitek Karas [MSFT]

    Tuesday, July 23, 2013 11:59 AM
    Moderator
  • Hi Vitek,

    Thanks for the clarification, it is a bit more clear now - the issue is due to some short commings of specifically Async scenarios on old .net frameworks.

    So basically we agree that in syncroneous scenarios the current implementation has a performance issue that could/should be fixed. I hope the right person from MSFT picks this up and put it in the bug base, or just fixes it.

    FYI, my increasing server pagesize alogorithm has a built in maximum, I may also make the page size/numbers of entities returned from one given page depend on the total processing time of the request. Some thing like "no matter what I am not going to spend more than X seconds on any given request" - if this happens the server will just finish sending more entities and send the skiptoken for getting the next page.

    Thanks again

    Uffe

    Tuesday, July 23, 2013 12:35 PM
  • The time limit on page is definitely a good solution. I've seen other services use that as well.

    I sent this question to the internal product team alias, for better visibility.

    Thanks,


    Vitek Karas [MSFT]

    Tuesday, July 23, 2013 1:35 PM
    Moderator
  • Hey Uffe,

    We have opened a bug on this performance issue.

    Thanks,

    Chris Robinson - WCF Data Services Team


    This posting is provided "AS IS" with no warranties, and confers no rights.

    Tuesday, July 23, 2013 3:55 PM
    Moderator
  • Thanks Chris. Looking forward to a fix. Uffe.
    Thursday, July 25, 2013 3:02 PM
  • I wouldn't call this a bug, I would call this a streaming feature. We would need to rewrite our usage of ODataLib within WCF Data Service Client to use the async apis everywhere whenever we use async and use sync apis when we use the sync version of http. Likely doing this would eliminate the issue. This is too much work to do for 5.6 especially given we are extremely close to shipping our 5.6 bits.

    We are working on version 6.0. We are keeping this issue in mind. But we have a lot on our plates for 6.0 to move OData to the v4.0 of the protocol so making these changes there is pretty low on our overall priority scale. The team really appreciates bugs like this that you are reporting so please keep reporting them.

    Thanks,

    Chris Robinson - WCF Data Services


    This posting is provided "AS IS" with no warranties, and confers no rights.

    Monday, August 12, 2013 4:17 PM
    Moderator
  • Also please contact me at christro at Microsoft dot com for any further concerns. We just don't see a quick fix for this issue at all.

    Thanks,

    Chris Robinson - WCF Data Services


    This posting is provided "AS IS" with no warranties, and confers no rights.

    Monday, August 12, 2013 4:22 PM
    Moderator