locked
Refresh entities exception: Recursion reached allowed limit: '100'. RRS feed

  • Question

  • On the client side I have sometimes to refresh all the entities, lets say of type T, so I have method which uses the System.Linq.Expressions.Expression to build an expression tree with all ids of those entities:

    //Getting all the entities to refresh:
    var entities = this.GetTheEntitiesToRefresh("this method doesn't exist");
    //Getting the property info of the primary key for example "Id"
    PropertyInfo pi = typeof(T).GetProperty("Id");
    //Creating an array of expressions of the type "id=1","id=3","id=33", etc
    var equals = entities.Select(e=>(System.Linq.Expressions.Expression)System.Linq.Expressions.Expression.Equal(System.Linq.Expressions.Expression.MakeMemberAccess(pe, pi), System.Linq.Expressions.Expression.Constant(pi.GetValue(e, null), pi.PropertyType)));
    //Joining together the expression with the or opertaor: "id=1 or id=3 or id=33 ... etc"
    var body = equals.Aggregate<System.Linq.Expressions.Expression>((accumulate, equal) => System.Linq.Expressions.Expression.Or(accumulate, equal));
    //Creating the lambda expression to use in the where clause
    Expression<Func<T, bool>> le = System.Linq.Expressions.Expression.Lambda<Func<T, bool>>(body, pe);
    //Creating the Data Service query
    DataServiceQuery<T> query = this.Service.TypeTSet.Where(le);
    //Refreshing entities:
    query.BeginExecute(....);

    And it works ... for a small number of entities, the problem is not a too long uri, instead the data service server throw an exception for too much recurisons (over 100). I know that I can configure it but if i take a look at the stack trace I see the following:

    [Stack Tace Bottom]

    And if I take a look to the generated uri I see the reason:

    http://localhost/DataService.svc/TypeTSet()?$filter=(((((Id eq 1) or (id eq 3)) or (id eq 33)) or ....

    If I remove all the brackets the service works without problem:

    http://localhost/DataService.svc/TypeTSet()?$filter=Id eq 1 or id eq 3 or id eq 33 or ....

    Now my question: This is a problem of Data Service, how it convert the expression tree to uri (with all those brackets) or is a problem of how I create the expression tree? It's possible to fix it and have a simpler uri which doesn't generate so many recursions on the server side?

    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.RecurseEnter()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalOr()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseParenExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParsePrimaryStart()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParsePrimary()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseUnary()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseMultiplicative()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseAdditive()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseComparison()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalAnd()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalOr()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseParenExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParsePrimaryStart()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParsePrimary()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseUnary()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseMultiplicative()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseAdditive()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseComparison()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalAnd()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalOr()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseParenExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParsePrimaryStart()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParsePrimary()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseUnary()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseMultiplicative()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseAdditive()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseComparison()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalAnd()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalOr()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseParenExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParsePrimaryStart()   
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParsePrimary()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseUnary()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseMultiplicative()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseAdditive()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseComparison()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalAnd()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalOr()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseParenExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParsePrimaryStart()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParsePrimary()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseUnary()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseMultiplicative()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseAdditive()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseComparison()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalAnd()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalOr()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseParenExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParsePrimaryStart()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParsePrimary()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseUnary()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseMultiplicative()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseAdditive()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseComparison()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalAnd()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalOr()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseParenExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParsePrimaryStart()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParsePrimary()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseUnary()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseMultiplicative()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseAdditive()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseComparison()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalAnd()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalOr()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseParenExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParsePrimaryStart()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParsePrimary()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseUnary()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseMultiplicative()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseAdditive()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseComparison()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalAnd()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalOr()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseParenExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParsePrimaryStart()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParsePrimary()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseUnary()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseMultiplicative()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseAdditive()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseComparison()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalAnd()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalOr()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseParenExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParsePrimaryStart()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParsePrimary()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseUnary()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseMultiplicative()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseAdditive()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseComparison()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalAnd()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalOr()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseParenExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParsePrimaryStart()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParsePrimary()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseUnary()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseMultiplicative()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseAdditive()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseComparison()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalAnd()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalOr()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseParenExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParsePrimaryStart()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParsePrimary()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseUnary()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseMultiplicative()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseAdditive()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseComparison()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalAnd()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalOr()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseParenExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParsePrimaryStart()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParsePrimary()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseUnary()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseMultiplicative()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseAdditive()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseComparison()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalAnd()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalOr()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseParenExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParsePrimaryStart()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParsePrimary()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseUnary()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseMultiplicative()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseAdditive()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseComparison()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalAnd()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseLogicalOr()   
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseExpression()    
    at System.Data.Services.Parsing.RequestQueryParser.ExpressionParser.ParseWhere()    
    at System.Data.Services.Parsing.RequestQueryParser.ParseLambdaForWhere(IDataService service, Type typeForIt, String expression)    
    at System.Data.Services.Parsing.RequestQueryParser.Where(IDataService service, IQueryable source, String predicate)    
    at System.Data.Services.RequestQueryProcessor.ProcessFilter()    
    at System.Data.Services.RequestQueryProcessor.ProcessQuery()    
    at System.Data.Services.RequestQueryProcessor.ProcessQuery(IDataService service, RequestDescription description)    
    at System.Data.Services.RequestUriProcessor.ProcessRequestUri(Uri absoluteRequestUri, IDataService service)    
    at System.Data.Services.DataService`1.ProcessIncomingRequestUriAndCacheHeaders()    
    at System.Data.Services.DataService`1.HandleRequest()

    Tuesday, March 30, 2010 2:16 PM

Answers

  • Thanks for the reply

    I know that the uri length could become a problem, but for the moment the number of entities to refresh is about 20-30, so I can be done also in one uri/request without problems. At the moment I have fixed it in the way that I generate the uri for the request without the brackets:

     

    //Getting all the entities to refresh:
    var entities = this.GetTheEntitiesToRefresh("this method doesn't exist");
    //Container for the equals expressions
    List<string> list = new List<string>();
    //Getting the property info of the primary key for example "Id"
    PropertyInfo pi = typeof(T).GetProperty("id");
    //Creating the equals expressions of the type "id eq 1","id eq 3","id eq 33", etc
    foreach (T entity in entities) list.Add(string.Format("{0} eq {1}", pi.Name, pi.GetValue(entity, null)));
    //Creating the url like: http://localhost/DataService.svc/TypeTSet()?$filter=id eq 1 or id eq 3 or id eq 33 ...
    Uri uri = new Uri(this.Service.TypeTSet.ToString() + "?$filter=" + string.Join(" or ", list.ToArray()));
    //Refreshing entities:
    this.Service.BeginExecute<T>(uri, ...);

     

    If the number of entities grows I can always split the request in smaller one and put them together in one batch request.

    Wednesday, March 31, 2010 7:31 AM

All replies

  • Hi,

    There might be some way to make it handle more. Basically the expression you generate is a tree, the server walks that tree using recursion. So you just need to make the tree shallow. So instead of (a and (b and (c and d))) use something like ((a and b) and (c and d)) and so on.

    But ultimately this will eventually run either into the URI limit or the recursion limit. It definitely doesn't scale well.

    I would consider some other options, for example:

    - Use batching - use batching to issue many small requests (maybe as small as one per entity). On the netwrok it's still going to be one request, you will pay some overhead on the server (As it has to unwrap the batch and treat each query in it separately) and on the DB (you would get multiple SQL statements, one for each query in the batch). But ultimately this might be worth it. On the client the overhead is not that big.

    - Use "custom" batching (so to say) - simply break it into multiple queries in your code - for example one query per 50 entities or so. In any case the query is one thing, but the response you get back is another. If the response is to contain 100 entities, it's going to be big, for 500 it's going to be huge.

    - Define a service operation. This is a bit hacky, but... You could define a service operation which takes for example a string parameter. In that string you pass in the list of IDs you need (some way) and in the service operation you turn that into some better SQL thing than a huge WHERE clause. One advantage is that in the other cases even if you somehow make the query work you're going to end up with a huge WHERE clause in the DB and the DB will take some time to process that as well.

    I would probably preffer the "custom" batching maybe combined with the batching request maybe not, where on the client you only issue queries for limited number of entities each time around. From the server point of view this best as you want each request to be fast, but you can handle many requests (makes the server much easier to handle multiple users at the same time). From the client you get to decide which entities you need and when. You also keep the payload size to reasonable numbers (usually production servers will have some kind of server driven paging turned on anyway, so you might not get the response to your huge query in one go anyway).

    Thanks,


    Vitek Karas [MSFT]
    Tuesday, March 30, 2010 5:46 PM
    Moderator
  • Thanks for the reply

    I know that the uri length could become a problem, but for the moment the number of entities to refresh is about 20-30, so I can be done also in one uri/request without problems. At the moment I have fixed it in the way that I generate the uri for the request without the brackets:

     

    //Getting all the entities to refresh:
    var entities = this.GetTheEntitiesToRefresh("this method doesn't exist");
    //Container for the equals expressions
    List<string> list = new List<string>();
    //Getting the property info of the primary key for example "Id"
    PropertyInfo pi = typeof(T).GetProperty("id");
    //Creating the equals expressions of the type "id eq 1","id eq 3","id eq 33", etc
    foreach (T entity in entities) list.Add(string.Format("{0} eq {1}", pi.Name, pi.GetValue(entity, null)));
    //Creating the url like: http://localhost/DataService.svc/TypeTSet()?$filter=id eq 1 or id eq 3 or id eq 33 ...
    Uri uri = new Uri(this.Service.TypeTSet.ToString() + "?$filter=" + string.Join(" or ", list.ToArray()));
    //Refreshing entities:
    this.Service.BeginExecute<T>(uri, ...);

     

    If the number of entities grows I can always split the request in smaller one and put them together in one batch request.

    Wednesday, March 31, 2010 7:31 AM