locked
WCF Data Services and EF5 auto-compiled query support with parameterized query RRS feed

  • Question

  • Hi,

    EF5 introduced auto-compiled query, this feature dramatically improve performance of Entity Framework.

    When we use WCF Data Service over an EF5 context, we automatically benefit from this feature. However it is possible to improve the way WCF Data Service build it expression tree to better use the auto-compiled query feature.

    Imagine we have the following query :

    http://localhost/WebApplication6/DataService.svc/Products(1)

    WCF Data Service will translate it to the following expression tree :

    .Call System.Linq.Queryable.Where(
        .Constant<System.Linq.EnumerableQuery`1[WebApplication6.Product]>(WebApplication6.Product[]),
        '(.Lambda #Lambda1<System.Func`2[WebApplication6.Product,System.Boolean]>))
    
    .Lambda #Lambda1<System.Func`2[WebApplication6.Product,System.Boolean]>(WebApplication6.Product $element) {
        $element.ProductId == 1
    }

    This expression tree will be compiled by EF5 (great!).
    However, if we query for another productId, WCF Data Service will generate a new expression tree (the only difference will be the constant value). When EF5 will verify if it has already compiled this expression tree, it won't find it and think that this expression is a new one, so it will compiled this new expression tree.

      => Each time we query for a new ProductId, EF5 will compiled a new expression tree (and cache it) :(

    EF5 can compiled these expression trees once if they used a ParameterExpression instead of ConstantExpression.

    Does the WCF Data Services team has planned to used ParameterExpression instead of ConstantExpression ? If no, pleased could you planned it ?I think it will boost a lot of WCF Data Services Scenario :-)

    By the way, I think it should be possible to use an ExpressionVisitor to rewrite the query and use parameterExpression instead of ConstantExpression. I didn't try it yet.

    Thanks,

    Cyril DURAND 



    Thursday, August 30, 2012 11:17 PM

All replies

  • Hi,

    You can use this site: http://data.uservoice.com/forums/72027-wcf-data-services-feature-suggestions to add a new feature request and let other people vote on it. It will help us determine what are the high priority missing features to implement.

    Thanks,


    Vitek Karas [MSFT]

    Friday, August 31, 2012 7:12 AM
    Moderator
  • Hi,

    Thanks for your reply.

    I suggest this idea here : http://data.uservoice.com/forums/72027-wcf-data-services-feature-suggestions/suggestions/3121561-use-parameterexpression-to-improve-performance-wit

    I'm not sure there will be a lot of vote, this idea is pretty technical and require to understand correctly how WCF Data Services works internally.

    I will try to make a visitor and post my result if I obtain something good.

    Thanks,

    Cyril DURAND


    Atteint de JavaScriptite aiguë

    Friday, August 31, 2012 8:27 AM
  • Hi,

    I did a visitor that extract constant to parameter (see code at bottom, it's very simple). Then, I wrote a sample DbSetWrapper based on the following code http://stackoverflow.com/questions/1839901/how-to-wrap-entity-framework-to-intercept-the-linq-expression-just-before-execut.
    When I use a DbContext, I can expose my DbSetWrapper and use it in my application. The query are correctly parameterized and compiled.

    WCF Data Services don't use exposed IDbSet on DbContext, it will create its ObjectQuery by calling the CreateQuery method on the underlying ObjectContext.
    I didn't find a way to modify the Expression generated by WCF Data Services.
    Is there any way to do it ?

    I know that I can implement IDataServiceMetadataProvider, IDataServiceQueryProvider, etc. but it's a really complicated just to have access to the expression tree.

    Parameterization will have a big performance impact. On my scenario, the first time I execute the query, it takes about 700ms, then 80ms. User don't access to the same item but will browse to many item : every time he access a new item, the query will take 700ms instead of 80ms.

    Please consider using parameter or let us easily access the underlying expression.

    Thanks,

    Cyril DURAND

    /// <summary> /// Will replace constant by parameter. It works by wrapping native constant by tuple constant. /// Example .Where(o => o.Id == 4) will be replaced by .Where(o => o.Id == new Tuple&lt;Int32&gt;(4).Item1.
    /// With such an expression EF will parameterize the constant /// </summary> internal class ConstantParameterizerVisitor : ExpressionVisitor { private readonly static Type[] _parameterizableTypes = new Type[] { typeof(Boolean), typeof(Byte), typeof(DateTime), typeof(DateTimeOffset), typeof(Decimal), typeof(Double), typeof(Single), typeof(Guid), typeof(Int16), typeof(Int32), typeof(Int64), typeof(SByte), typeof(Nullable<Boolean>), typeof(Nullable<Byte>), typeof(Nullable<DateTime>), typeof(Nullable<DateTimeOffset>), typeof(Nullable<Decimal>), typeof(Nullable<Double>), typeof(Nullable<Single>), typeof(Nullable<Guid>), typeof(Nullable<Int16>), typeof(Nullable<Int32>), typeof(Nullable<Int64>), typeof(Nullable<SByte>), typeof(String) }; private ConstantParameterizerVisitor() { } public static Expression Parameterize(Expression expression) { expression = new ConstantParameterizerVisitor().Visit(expression); return expression; } protected override Expression VisitConstant(ConstantExpression node) { if (ConstantParameterizerVisitor._parameterizableTypes.Contains(node.Type)) { Type wrappedType = typeof(Tuple<>).MakeGenericType(node.Type); Object wrappedObject = Activator.CreateInstance(wrappedType, new Object[] { node.Value }); Expression visitedExpression = Expression.Property(Expression.Constant(wrappedObject), "Item1"); return visitedExpression; } return base.VisitConstant(node); } }


    Atteint de JavaScriptite aiguë

    Monday, September 10, 2012 12:23 PM
  • I've voted. Doing differently is just crazy in any data-intensive system

    Roman Koreshkov, systems architect

    Tuesday, October 2, 2012 10:38 AM
  • We are planning to make the EF based provider public. Once we have done that, it would be very easy to do this kind of customization. You can override GetQueryRoot method and return your DbSetWrapper and then have the visitor translate the constant expression into the parameter expression.

    Hopefully we do this feature soon. Maybe you should vote on that feature also, since that is a very general purpose feature that will enable so many other scenarios also.

    Thanks

    Pratik


    This posting is provided "AS IS" with no warranties, and confers no rights.

    Friday, October 5, 2012 5:56 PM
    Moderator
  • Hi Patrik,

    Thanks for the reply.

    When you say :

    We are planning to make the EF based provider public.

    Do you mean that you will push source code on codeplex ? or mark the class as public instead of internal ? I hope you will mark the class as public and add more virtual methods :)

    Do you know when is it planned ?

    About the initial problem, why don't you parameterize query ? it seems to be pretty simple. Not a lot of user understand how EF and WCF Data Services works. If you do it, everybody will be able to benefit from the great improvement.

    Thanks,


    Atteint de JavaScriptite aiguë

    Monday, October 8, 2012 8:49 AM
  • I meant making the class public and making all the IDataServiceMetadataProvider, IDataServiceQueryProvider and IDataServiceUpdateProvider methods virtual, so that folks can override and intercept the call. I think this is a very general feature and folks can do a lot of things once we open up the EF provider in this manner.

    About parameterizing the query, we would love to do that - its a breaking change (unless we do it only for EF). The way we are leaning more and more is that we should treat EF more and more as a general provider and let folks write simple visitor to make change the expression tree. If this becomes very popular, we can ship this visitor (one that parameterizes the query) inside the EF provider.

    About planned, we have it in our backlog and we have been trying to get this done - this is very high in our priority list, so hopefully later this year or early next year would be my guess. No promises though.

    Thanks

    Pratik


    This posting is provided "AS IS" with no warranties, and confers no rights.

    Tuesday, October 9, 2012 6:44 PM
    Moderator
  • Hi Patrik,

    Do you have any update on making the EF provider public ?

    Is there a release date, or public/internal beta we could try ?

    Thanks,

    Cyril DURAND


    Atteint de JavaScriptite aiguë

    Tuesday, January 22, 2013 11:01 AM
  • Hello Pratik,

    I share Cyril's concern. I would like to make extensive use of WCF Data Services in our code base, but not having parametrized queries is a big performance hit. I have to stick with Entity Framework + DTOs + traditional WCF methods for the time being. Any word on when do you expect to open the code? It looks like the last two releases (5.2, 5.3) are just consolidating WCF Data Services with WebAPI. Do you a have a roadmap?

    Cyril, I already voted for your suggestion, but I'm afraid not enough people understand the technical implications. This technology is impressive and solves a lot of data access scenarios in a very elegant way, but the performance has to be equal (or better) than existing options.

    Saturday, February 9, 2013 3:11 AM
  • I also voted for this feature. I would hazard a guess and to say this is a deal-breaking issue for WCF Data Services for any high-volume enterprise level data service. This really should be the default, with a fallback to the old behavior for legacy applications for which this might be a breaking change. A significant percentage of the overhead in fulfilling WCF Data Service queries on the SQL Server side is in compiling these queries, so having it compile a new query for each page of a list, or each item in the list, each user-customizable list filter, etc... is simply not scalable enough to meet the demands of non-trivial server loads, and I would also classify this as a denial of service vulnerability, as an attacker could then easily automate DOS queries in such a way as to never use a precompiled query and thus multiply the effectiveness of the attack (both by making their requests take longer, and by thrashing the plan cache, making legitimate users' requests take longer too). This is not theoretical. Please escalate this. Thanks!

    • Edited by JeroMiya Saturday, February 9, 2013 5:25 PM
    Saturday, February 9, 2013 5:07 PM
  • Hello,

    Any official  word about this or a follow up?


    • Edited by OmoraRA Sunday, April 21, 2013 6:23 AM
    Sunday, April 21, 2013 6:22 AM
  • Sorry for the delay in answer. Finally, we are close to done on this feature and we will be releasing this as part of our next release (5.5 RTM). The beta for this release should be out soon (in about 1.5 weeks or so), with RTM around third week of May'13.

    As I said earlier in this thread, this will make the EF provider public so that folks can wrap the IQueryable and write the visitor to solve the issue mentioned in this thread.

    Thanks

    Pratik


    This posting is provided "AS IS" with no warranties, and confers no rights.


    Thursday, April 25, 2013 12:36 PM
    Moderator
  • Sorry for the delay in answer. Finally, we are close to done on this feature and we will be releasing this as part of our next release (5.5 RTM). The beta for this release should be out soon (in about 1.5 weeks or so), with RTM around third week of May'13.

    As I said earlier in this thread, this will make the EF provider public so that folks can wrap the IQueryable and write the visitor to solve the issue mentioned in this thread.

    Thanks

    Pratik


    This posting is provided "AS IS" with no warranties, and confers no rights.


    sorry to ask before release but could someone come back to this thread when we get 5.5 and give an example of how this will work? Very interested in the solution. Downloaded the aplha but apparently it's not there yet.

    Also please bring back odata.codeplex.com. We played a lot with it and got some pretty cool things. Now we're planning to move the server to wcf 5.5 but keep the client on 4 because we have too many things which depend on changing the code here and there.

    Friday, April 26, 2013 8:27 PM
  • Thanks for the update Pratik, this is really great news.

    I'm about to start a new project and I really want to include WCF Data Services. This was a deal breaker, but I'm glad there will be a way to fix it.

    Thursday, May 2, 2013 5:47 AM
  • great work so far :) managed to put it all together, and made something like this (work in progress).

    public class QueryTranslator<T> : IOrderedQueryable<T>
        {
            private Expression expression = null;
            private QueryTranslatorProvider<T> provider = null;
    
            public QueryTranslator(IQueryable source)
            {
                expression = Expression.Constant(this);
                provider = new QueryTranslatorProvider<T>(source);
            }
    
            public QueryTranslator(IQueryable source, Expression e)
            {
                if (e == null) throw new ArgumentNullException("e");
                expression = e;
                provider = new QueryTranslatorProvider<T>(source);
            }
    
            public IEnumerator<T> GetEnumerator()
            {
                return ((IEnumerable<T>)provider.ExecuteEnumerable(this.expression)).GetEnumerator();
            }
    
            System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
            {
                return provider.ExecuteEnumerable(this.expression).GetEnumerator();
            }
    
            public QueryTranslator<T> Include(String path)
            {
                ObjectQuery<T> possibleObjectQuery = provider.source as ObjectQuery<T>;
                if (possibleObjectQuery != null)
                {
                    return new QueryTranslator<T>(possibleObjectQuery.Include(path));
                }
                else
                {
                    throw new InvalidOperationException("The Include should only happen at the beginning of a LINQ expression");
                }
            }
    
            public Type ElementType
            {
                get { return typeof(T); }
            }
    
            public Expression Expression
            {
                get { return expression; }
            }
    
            public IQueryProvider Provider
            {
                get { return provider; }
            }
        } 
    
        public class QueryTranslatorProvider<T> : ExpressionVisitor, IQueryProvider
        {
            internal IQueryable source;
    
    
    
            public QueryTranslatorProvider(IQueryable source)
            {
                if (source == null) throw new ArgumentNullException("source");
                this.source = source;
            }
    
            public IQueryable<TElement> CreateQuery<TElement>(Expression expression)
            {
                if (expression == null) throw new ArgumentNullException("expression");
    
                return new QueryTranslator<TElement>(source, expression) as IQueryable<TElement>;
            }
    
            public IQueryable CreateQuery(Expression expression)
            {
                if (expression == null) throw new ArgumentNullException("expression");
                Type elementType = expression.Type.GetGenericArguments().First();
                IQueryable result = (IQueryable)Activator.CreateInstance(typeof(QueryTranslator<>).MakeGenericType(elementType),
                    new object[] { source, expression });
                return result;
            }
    
            public TResult Execute<TResult>(Expression expression)
            {
                if (expression == null) throw new ArgumentNullException("expression");
                object result = (this as IQueryProvider).Execute(expression);
                return (TResult)result;
            }
    
            public object Execute(Expression expression)
            {
                if (expression == null) throw new ArgumentNullException("expression");
    
                Expression translated = this.Visit(expression);
                return source.Provider.Execute(translated);
            }
    
            internal IEnumerable ExecuteEnumerable(Expression expression)
            {
                if (expression == null) throw new ArgumentNullException("expression");
                Boolean debug_Var = false;
                if (debug_Var)
                {
                    return source.Provider.CreateQuery(expression);
                }
                else
                {
                    Expression translated = this.Visit(expression);
                    return source.Provider.CreateQuery(translated);
                }
            }
    
            #region Visitors
    
    
    
            protected override Expression VisitConstant(ConstantExpression c)
            {
                // fix up the Expression tree to work with EF again
                if (ConstantParameterizerVisitor._parameterizableTypes.Contains(c.Type))
                {
                    Type wrappedType = typeof(Tuple<>).MakeGenericType(c.Type);
                    Object wrappedObject = Activator.CreateInstance(wrappedType, new Object[] { c.Value });
                    Expression visitedExpression = Expression.Property(Expression.Constant(wrappedObject), "Item1");
                    return visitedExpression;
                }
                if (c.Type == typeof(QueryTranslator<T>))
                {
                    return source.Expression;
                }
                else
                {
                    if (c.Type.Name == typeof(QueryTranslator<T>).Name)
                    {
                        return source.Expression;
                    }
                    else
                    {
                        return base.VisitConstant(c);
                    }
                }
            }
            #endregion
        }
    
        internal class ConstantParameterizerVisitor : ExpressionVisitor
        {
            public readonly static Type[] _parameterizableTypes = new Type[] { 
                    typeof(Boolean), 
                    typeof(Byte), 
                    typeof(DateTime), 
                    typeof(DateTimeOffset), 
                    typeof(Decimal), 
                    typeof(Double), 
                    typeof(Single), 
                    typeof(Guid), 
                    typeof(Int16), 
                    typeof(Int32), 
                    typeof(Int64), 
                    typeof(SByte), 
                    typeof(Nullable<Boolean>), 
                    typeof(Nullable<Byte>), 
                    typeof(Nullable<DateTime>), 
                    typeof(Nullable<DateTimeOffset>), 
                    typeof(Nullable<Decimal>), 
                    typeof(Nullable<Double>), 
                    typeof(Nullable<Single>), 
                    typeof(Nullable<Guid>), 
                    typeof(Nullable<Int16>), 
                    typeof(Nullable<Int32>), 
                    typeof(Nullable<Int64>), 
                    typeof(Nullable<SByte>), 
                    typeof(String) 
                };
    
    
            private ConstantParameterizerVisitor()
            { }
    
            public static Expression Parameterize(Expression expression)
            {
                expression = new ConstantParameterizerVisitor().Visit(expression);
                return expression;
            }
    
            protected override Expression VisitConstant(ConstantExpression node)
            {
                if (ConstantParameterizerVisitor._parameterizableTypes.Contains(node.Type))
                {
                    Type wrappedType = typeof(Tuple<>).MakeGenericType(node.Type);
                    Object wrappedObject = Activator.CreateInstance(wrappedType, new Object[] { node.Value });
                    Expression visitedExpression = Expression.Property(Expression.Constant(wrappedObject), "Item1");
                    return visitedExpression;
                }
                return base.VisitConstant(node);
            }
        }
    
    
        public abstract class ExpressionVisitor
        {
            protected ExpressionVisitor()
            {
            }
    
            protected virtual Expression Visit(Expression exp)
            {
                if (exp == null)
                    return exp;
                switch (exp.NodeType)
                {
                    case ExpressionType.Negate:
                    case ExpressionType.NegateChecked:
                    case ExpressionType.Not:
                    case ExpressionType.Convert:
                    case ExpressionType.ConvertChecked:
                    case ExpressionType.ArrayLength:
                    case ExpressionType.Quote:
                    case ExpressionType.TypeAs:
                        return this.VisitUnary((UnaryExpression)exp);
                    case ExpressionType.Add:
                    case ExpressionType.AddChecked:
                    case ExpressionType.Subtract:
                    case ExpressionType.SubtractChecked:
                    case ExpressionType.Multiply:
                    case ExpressionType.MultiplyChecked:
                    case ExpressionType.Divide:
                    case ExpressionType.Modulo:
                    case ExpressionType.And:
                    case ExpressionType.AndAlso:
                    case ExpressionType.Or:
                    case ExpressionType.OrElse:
                    case ExpressionType.LessThan:
                    case ExpressionType.LessThanOrEqual:
                    case ExpressionType.GreaterThan:
                    case ExpressionType.GreaterThanOrEqual:
                    case ExpressionType.Equal:
                    case ExpressionType.NotEqual:
                    case ExpressionType.Coalesce:
                    case ExpressionType.ArrayIndex:
                    case ExpressionType.RightShift:
                    case ExpressionType.LeftShift:
                    case ExpressionType.ExclusiveOr:
                        return this.VisitBinary((BinaryExpression)exp);
                    case ExpressionType.TypeIs:
                        return this.VisitTypeIs((TypeBinaryExpression)exp);
                    case ExpressionType.Conditional:
                        return this.VisitConditional((ConditionalExpression)exp);
                    case ExpressionType.Constant:
                        return this.VisitConstant((ConstantExpression)exp);
                    case ExpressionType.Parameter:
                        return this.VisitParameter((ParameterExpression)exp);
                    case ExpressionType.MemberAccess:
                        return this.VisitMemberAccess((MemberExpression)exp);
                    case ExpressionType.Call:
                        return this.VisitMethodCall((MethodCallExpression)exp);
                    case ExpressionType.Lambda:
                        return this.VisitLambda((LambdaExpression)exp);
                    case ExpressionType.New:
                        return this.VisitNew((NewExpression)exp);
                    case ExpressionType.NewArrayInit:
                    case ExpressionType.NewArrayBounds:
                        return this.VisitNewArray((NewArrayExpression)exp);
                    case ExpressionType.Invoke:
                        return this.VisitInvocation((InvocationExpression)exp);
                    case ExpressionType.MemberInit:
                        return this.VisitMemberInit((MemberInitExpression)exp);
                    case ExpressionType.ListInit:
                        return this.VisitListInit((ListInitExpression)exp);
                    default:
                        throw new Exception(string.Format("Unhandled expression type: '{0}'", exp.NodeType));
                }
            }
    
            protected virtual MemberBinding VisitBinding(MemberBinding binding)
            {
                switch (binding.BindingType)
                {
                    case MemberBindingType.Assignment:
                        return this.VisitMemberAssignment((MemberAssignment)binding);
                    case MemberBindingType.MemberBinding:
                        return this.VisitMemberMemberBinding((MemberMemberBinding)binding);
                    case MemberBindingType.ListBinding:
                        return this.VisitMemberListBinding((MemberListBinding)binding);
                    default:
                        throw new Exception(string.Format("Unhandled binding type '{0}'", binding.BindingType));
                }
            }
    
            protected virtual ElementInit VisitElementInitializer(ElementInit initializer)
            {
                ReadOnlyCollection<Expression> arguments = this.VisitExpressionList(initializer.Arguments);
                if (arguments != initializer.Arguments)
                {
                    return Expression.ElementInit(initializer.AddMethod, arguments);
                }
                return initializer;
            }
    
            protected virtual Expression VisitUnary(UnaryExpression u)
            {
                Expression operand = this.Visit(u.Operand);
                if (operand != u.Operand)
                {
                    return Expression.MakeUnary(u.NodeType, operand, u.Type, u.Method);
                }
                return u;
            }
    
            protected virtual Expression VisitBinary(BinaryExpression b)
            {
                Expression left = this.Visit(b.Left);
                Expression right = this.Visit(b.Right);
                Expression conversion = this.Visit(b.Conversion);
                if (left != b.Left || right != b.Right || conversion != b.Conversion)
                {
                    if (b.NodeType == ExpressionType.Coalesce && b.Conversion != null)
                        return Expression.Coalesce(left, right, conversion as LambdaExpression);
                    else
                        return Expression.MakeBinary(b.NodeType, left, right, b.IsLiftedToNull, b.Method);
                }
                return b;
            }
    
            protected virtual Expression VisitTypeIs(TypeBinaryExpression b)
            {
                Expression expr = this.Visit(b.Expression);
                if (expr != b.Expression)
                {
                    return Expression.TypeIs(expr, b.TypeOperand);
                }
                return b;
            }
    
            protected virtual Expression VisitConstant(ConstantExpression c)
            {
                return c;
            }
    
            protected virtual Expression VisitConditional(ConditionalExpression c)
            {
                Expression test = this.Visit(c.Test);
                Expression ifTrue = this.Visit(c.IfTrue);
                Expression ifFalse = this.Visit(c.IfFalse);
                if (test != c.Test || ifTrue != c.IfTrue || ifFalse != c.IfFalse)
                {
                    return Expression.Condition(test, ifTrue, ifFalse);
                }
                return c;
            }
    
            protected virtual Expression VisitParameter(ParameterExpression p)
            {
                return p;
            }
    
            protected virtual Expression VisitMemberAccess(MemberExpression m)
            {
                Expression exp = this.Visit(m.Expression);
                if (exp != m.Expression)
                {
                    return Expression.MakeMemberAccess(exp, m.Member);
                }
                return m;
            }
    
            protected virtual Expression VisitMethodCall(MethodCallExpression m)
            {
                Expression obj = this.Visit(m.Object);
                IEnumerable<Expression> args = this.VisitExpressionList(m.Arguments);
                if (obj != m.Object || args != m.Arguments)
                {
                    return Expression.Call(obj, m.Method, args);
                }
                return m;
            }
    
            protected virtual ReadOnlyCollection<Expression> VisitExpressionList(ReadOnlyCollection<Expression> original)
            {
                List<Expression> list = null;
                for (int i = 0, n = original.Count; i < n; i++)
                {
                    Expression p = this.Visit(original[i]);
                    if (list != null)
                    {
                        list.Add(p);
                    }
                    else if (p != original[i])
                    {
                        list = new List<Expression>(n);
                        for (int j = 0; j < i; j++)
                        {
                            list.Add(original[j]);
                        }
                        list.Add(p);
                    }
                }
                if (list != null)
                {
                    return list.AsReadOnly();
                }
                return original;
            }
    
            protected virtual MemberAssignment VisitMemberAssignment(MemberAssignment assignment)
            {
                Expression e = this.Visit(assignment.Expression);
                if (e != assignment.Expression)
                {
                    return Expression.Bind(assignment.Member, e);
                }
                return assignment;
            }
    
            protected virtual MemberMemberBinding VisitMemberMemberBinding(MemberMemberBinding binding)
            {
                IEnumerable<MemberBinding> bindings = this.VisitBindingList(binding.Bindings);
                if (bindings != binding.Bindings)
                {
                    return Expression.MemberBind(binding.Member, bindings);
                }
                return binding;
            }
    
            protected virtual MemberListBinding VisitMemberListBinding(MemberListBinding binding)
            {
                IEnumerable<ElementInit> initializers = this.VisitElementInitializerList(binding.Initializers);
                if (initializers != binding.Initializers)
                {
                    return Expression.ListBind(binding.Member, initializers);
                }
                return binding;
            }
    
            protected virtual IEnumerable<MemberBinding> VisitBindingList(ReadOnlyCollection<MemberBinding> original)
            {
                List<MemberBinding> list = null;
                for (int i = 0, n = original.Count; i < n; i++)
                {
                    MemberBinding b = this.VisitBinding(original[i]);
                    if (list != null)
                    {
                        list.Add(b);
                    }
                    else if (b != original[i])
                    {
                        list = new List<MemberBinding>(n);
                        for (int j = 0; j < i; j++)
                        {
                            list.Add(original[j]);
                        }
                        list.Add(b);
                    }
                }
                if (list != null)
                    return list;
                return original;
            }
    
            protected virtual IEnumerable<ElementInit> VisitElementInitializerList(ReadOnlyCollection<ElementInit> original)
            {
                List<ElementInit> list = null;
                for (int i = 0, n = original.Count; i < n; i++)
                {
                    ElementInit init = this.VisitElementInitializer(original[i]);
                    if (list != null)
                    {
                        list.Add(init);
                    }
                    else if (init != original[i])
                    {
                        list = new List<ElementInit>(n);
                        for (int j = 0; j < i; j++)
                        {
                            list.Add(original[j]);
                        }
                        list.Add(init);
                    }
                }
                if (list != null)
                    return list;
                return original;
            }
    
            protected virtual Expression VisitLambda(LambdaExpression lambda)
            {
                Expression body = this.Visit(lambda.Body);
                if (body != lambda.Body)
                {
                    return Expression.Lambda(lambda.Type, body, lambda.Parameters);
                }
                return lambda;
            }
    
            protected virtual NewExpression VisitNew(NewExpression nex)
            {
                IEnumerable<Expression> args = this.VisitExpressionList(nex.Arguments);
                if (args != nex.Arguments)
                {
                    if (nex.Members != null)
                        return Expression.New(nex.Constructor, args, nex.Members);
                    else
                        return Expression.New(nex.Constructor, args);
                }
                return nex;
            }
    
            protected virtual Expression VisitMemberInit(MemberInitExpression init)
            {
                NewExpression n = this.VisitNew(init.NewExpression);
                IEnumerable<MemberBinding> bindings = this.VisitBindingList(init.Bindings);
                if (n != init.NewExpression || bindings != init.Bindings)
                {
                    return Expression.MemberInit(n, bindings);
                }
                return init;
            }
    
            protected virtual Expression VisitListInit(ListInitExpression init)
            {
                NewExpression n = this.VisitNew(init.NewExpression);
                IEnumerable<ElementInit> initializers = this.VisitElementInitializerList(init.Initializers);
                if (n != init.NewExpression || initializers != init.Initializers)
                {
                    return Expression.ListInit(n, initializers);
                }
                return init;
            }
    
            protected virtual Expression VisitNewArray(NewArrayExpression na)
            {
                IEnumerable<Expression> exprs = this.VisitExpressionList(na.Expressions);
                if (exprs != na.Expressions)
                {
                    if (na.NodeType == ExpressionType.NewArrayInit)
                    {
                        return Expression.NewArrayInit(na.Type.GetElementType(), exprs);
                    }
                    else
                    {
                        return Expression.NewArrayBounds(na.Type.GetElementType(), exprs);
                    }
                }
                return na;
            }
    
            protected virtual Expression VisitInvocation(InvocationExpression iv)
            {
                IEnumerable<Expression> args = this.VisitExpressionList(iv.Arguments);
                Expression expr = this.Visit(iv.Expression);
                if (args != iv.Arguments || expr != iv.Expression)
                {
                    return Expression.Invoke(expr, args);
                }
                return iv;
            }
        }


    the results are very good performance wise (i would show you but i'm not approved yet to put images or urls)

    Implemented IServiceProvider in my service, created a derived class from EntityFrameworkDataServiceProvider, something like so:

     public class EFExt : EntityFrameworkDataServiceProvider
            {
                public EFExt(DataServiceProviderArgs args)
                    : base(args)
                {
                    
                }
                public override IQueryable GetQueryRootForResourceSet(ResourceSet resourceSet)
                {
                    var temp = base.GetQueryRootForResourceSet(resourceSet);
        			
                    if(typeof(ABCD_CLASS)==temp.ElementType)  return new QueryTranslator<ABCD_CLASS>(temp);
                    if(typeof(ECDF_CLASS)==temp.ElementType)  return new QueryTranslator<ECDF_CLASS>(temp);
    
                    return temp;
                }
            }


    i know, kinda stupid implementation but for querying it works and lightning fast.

    BUT! For inserting although I have no problems for updating and deleting I do, as in GetResource in ObjectContextServiceProvider it goes and casts it to an ObjectQuery (I peeked)

      public virtual object GetResource(IQueryable query, string fullTypeName)
            {
                WebUtil.CheckArgumentNull<IQueryable>(query, "query");
                ObjectQuery query2 = (ObjectQuery) query;
                query2.MergeOption = MergeOption.AppendOnly;
                object resource = null;
                foreach (object obj3 in (IEnumerable) query2)
                {
                    if (resource != null)
                    {
                        throw new InvalidOperationException(System.Data.Services.Strings.SingleResourceExpected);
                    }
                    resource = obj3;
                }
                if (resource != null)
                {
                    ResourceType resourceType = this.GetResourceType(resource);
                    if (resourceType == null)
                    {
                        throw new InvalidOperationException(System.Data.Services.Strings.ObjectContext_UnknownResourceTypeForClrType(resource.GetType().FullName));
                    }
                    if ((fullTypeName != null) && (resourceType.FullName != fullTypeName))
                    {
                        throw DataServiceException.CreateBadRequestError(System.Data.Services.Strings.TargetElementTypeOfTheUriSpecifiedDoesNotMatchWithTheExpectedType(resourceType.FullName, fullTypeName));
                    }
                }
                return resource;
            }

    and I get this error:

    Unable to cast object of type 'ABC_Service.QueryTranslator`1[ABC_Service.ABCD_CLASS]' to type 'System.Data.Objects.ObjectQuery'.


    so... any suggestions how to go about this would be greatly appreciated.

    also don't quote this post in this thread. the internet thanks you.



    • Edited by sf.petru Wednesday, May 29, 2013 8:07 PM
    Wednesday, May 29, 2013 8:03 PM
  • Hi,

    Thanks for binging it up.

    I have tried your code and tweak it a little bit. It works fine now for CRUD. It overall improved my test scenario by 25%.

        /// <summary>
        /// EFParameterizedQueryProvider
        /// </summary>
        class EFParameterizedQueryProvider : IQueryProvider
        {
            /// <summary>
            /// Cache the CreateEFParameterizedQuery generic methodinfo
            /// </summary>
            readonly static MethodInfo CreateEFParameterizedQueryMethod =
                typeof (EFParameterizedQueryProvider).GetMethod("CreateEFParameterizedQuery",
                    BindingFlags.Instance | BindingFlags.NonPublic);
    
            /// <summary>
            /// The underlying ef query provider
            /// </summary>
            readonly IQueryProvider underlyingQueryProvider;
    
            /// <summary>
            /// Create an instance of EFParameterizedQueryProvider
            /// </summary>
            /// <param name="underlyingQueryProvider">The underlying ef query providera</param>
            EFParameterizedQueryProvider(IQueryProvider underlyingQueryProvider)
            {
                this.underlyingQueryProvider = underlyingQueryProvider;
            }
    
            /// <summary>
            /// Craete a query from EF queryable
            /// </summary>
            /// <param name="underlyingQuery">The underlying EF query</param>
            /// <returns>The queryable EFParameterizedQuery</returns>
            public static IQueryable CreateQuery(IQueryable underlyingQuery)
            {
                var provider = new EFParameterizedQueryProvider(underlyingQuery.Provider);
                Type elementType = underlyingQuery.Expression.Type.GetQueryElementType();
                return (IQueryable)CreateEFParameterizedQueryMethod.MakeGenericMethod(elementType).Invoke(provider, new object[] { underlyingQuery.Expression, underlyingQuery });
            }
    
            /// <summary>
            /// Create a query from the expression
            /// </summary>
            /// <param name="expression">The expression</param>
            /// <returns>The queryable EFParameterizedQuery</returns>
            public IQueryable CreateQuery(Expression expression)
            {
                Type elementType = expression.Type.GetQueryElementType();
                return (IQueryable)CreateEFParameterizedQueryMethod.MakeGenericMethod(elementType).Invoke(this, new object[] { expression, null });
            }
    
            /// <summary>
            /// Create a query from the expression
            /// </summary>
            /// <typeparam name="TElement">The element type</typeparam>
            /// <param name="expression">The expression</param>
            /// <returns>The queryable EFParameterizedQuery</returns>
            public IQueryable<TElement> CreateQuery<TElement>(Expression expression)
            {
                return CreateEFParameterizedQuery<TElement>(expression, null);
            }
    
            /// <summary>
            /// Create a EFParameterizedQuery from the expression and original IQueryable if we have
            /// </summary>
            /// <typeparam name="TElement">The element type</typeparam>
            /// <param name="expression">The expression</param>
            /// <param name="queryable">The original IQueryable</param>
            /// <returns></returns>
            private EFParameterizedQuery<TElement> CreateEFParameterizedQuery<TElement>(Expression expression, IQueryable queryable)
            {
                var objectQuery = queryable as ObjectQuery<TElement> ??
                                  (ObjectQuery<TElement>)this.underlyingQueryProvider.CreateQuery<TElement>(expression);
                return new EFParameterizedQuery<TElement>(objectQuery, this);
            }
    
            /// <summary>
            /// Execute the expression
            /// </summary>
            /// <param name="expression">The expression</param>
            /// <returns>The result</returns>
            public object Execute(Expression expression)
            {
                Expression parameterizedExpression = new EFParameterizedExpressionVisitor().Parameterize(expression);
                if (typeof(IQueryable).IsAssignableFrom(expression.Type))
                {
                    return underlyingQueryProvider.CreateQuery(parameterizedExpression);
                }
                return underlyingQueryProvider.Execute(parameterizedExpression);
            }
    
            /// <summary>
            /// Execute the expression
            /// </summary>
            /// <typeparam name="TResult">The result type</typeparam>
            /// <param name="expression">The expression</param>
            /// <returns>The result</returns>
            public TResult Execute<TResult>(Expression expression)
            {
                return (TResult)Execute(expression);
            }
        }
    
        /// <summary>
        /// EF Parameterized Query
        /// </summary>
        abstract class EFParameterizedQuery
        {
            /// <summary>
            /// The original EF ObjectQuery
            /// </summary>
            protected readonly ObjectQuery objectQuery;
    
            /// <summary>
            /// Create an instance of type EFParameterizedQuery
            /// </summary>
            /// <param name="objectQuery">The original EF ObjectQuery</param>
            protected EFParameterizedQuery(ObjectQuery objectQuery)
            {
                this.objectQuery = objectQuery;
            }
    
            /// <summary>
            /// The original EF ObjectQuery
            /// </summary>
            public ObjectQuery ObjectQuery
            {
                get { return objectQuery; }
            }
        }
    
        /// <summary>
        /// EF Parameterized Query of type T
        /// </summary>
        /// <typeparam name="T"></typeparam>
        class EFParameterizedQuery<T> : EFParameterizedQuery, IOrderedQueryable<T>
        {
            /// <summary>
            /// The query provider
            /// </summary>
            readonly IQueryProvider queryProvider;
    
            /// <summary>
            /// Create an instance of type EFParameterizedQuery
            /// </summary>
            /// <param name="objectQuery">The original EF ObjectQuery</param>
            /// <param name="queryProvider">The query provider</param>
            public EFParameterizedQuery(ObjectQuery<T> objectQuery, IQueryProvider queryProvider)
                :base(objectQuery)
            {
                this.queryProvider = queryProvider;
            }
    
            /// <summary>
            /// GetEnumerator
            /// </summary>
            /// <returns>IEnumerator</returns>
            public IEnumerator<T> GetEnumerator()
            {
                return queryProvider.Execute<IEnumerable<T>>(Expression).GetEnumerator();
            }
    
            /// <summary>
            /// GetEnumerator
            /// </summary>
            /// <returns>IEnumerator</returns>
            IEnumerator IEnumerable.GetEnumerator()
            {
                return GetEnumerator();
            }
    
            /// <summary>
            /// Get the current expression
            /// </summary>
            public Expression Expression
            {
                get { return ((IQueryable)objectQuery).Expression; }
            }
    
            /// <summary>
            /// Get the element type
            /// </summary>
            public Type ElementType
            {
                get { return typeof(T); }
            }
    
            /// <summary>
            /// Get the query provider
            /// </summary>
            public IQueryProvider Provider
            {
                get { return queryProvider; }
            }
        }

    The usage is like

        public class EFProvider : EntityFrameworkDataServiceProvider
        {
            public EFProvider(EFService service, AstoriaDefaultServiceDBEntities container)
                : base(new DataServiceProviderArgs(service, container, null, false))
            {}
    
            public override IQueryable GetQueryRootForResourceSet(ResourceSet resourceSet)
            {
                return EFParameterizedQueryProvider.CreateQuery(base.GetQueryRootForResourceSet(resourceSet));
            }
    
            public override object GetResource(IQueryable query, string fullTypeName)
            {
                var efParameterizedQuery = query as EFParameterizedQuery;
                var objectQuery = efParameterizedQuery == null ? query : efParameterizedQuery.ObjectQuery;
                return base.GetResource(objectQuery, fullTypeName);
            } 
        }

    The EFParameterizedExpressionVisitor is similar to yours , so i didn't paste it again.

    Let me know if you have any other questions.

    Wednesday, June 19, 2013 10:25 PM
  • just one question.

    can't seem to find GetQueryElementType()

    Thursday, June 20, 2013 11:26 AM
  • Here you go.

    It is basically the same purpose as your one line code.

    Type elementType = expression.Type.GetGenericArguments().First();

        static class TypeExtension
        {
            public static Type GetQueryElementType(this Type type)
            {
                Type ienum = FindIEnumerable(type);
                if (ienum == null)
                {
                    return type;
                }
    
                return ienum.GetGenericArguments()[0];
            }
    
            static Type FindIEnumerable(Type seqType)
            {
                if (seqType == null || seqType == typeof(string))
                {
                    return null;
                }
    
                if (seqType.IsArray)
                {
                    return typeof(IEnumerable<>).MakeGenericType(seqType.GetElementType());
                }
    
                if (seqType.IsGenericType)
                {
                    foreach (Type arg in seqType.GetGenericArguments())
                    {
                        Type ienum = typeof(IEnumerable<>).MakeGenericType(arg);
                        if (ienum.IsAssignableFrom(seqType))
                        {
                            return ienum;
                        }
                    }
                }
    
                Type[] ifaces = seqType.GetInterfaces();
                if (ifaces.Length > 0)
                {
                    foreach (Type iface in ifaces)
                    {
                        Type ienum = FindIEnumerable(iface);
                        if (ienum != null)
                        {
                            return ienum;
                        }
                    }
                }
    
                if (seqType.BaseType != null && seqType.BaseType != typeof(object))
                {
                    return FindIEnumerable(seqType.BaseType);
                }
    
                return null;
            }
        }

    Thursday, June 20, 2013 6:35 PM
  • We just published the Entity Framework Parameterized Query sample.

    http://code.msdn.microsoft.com/Entity-Framework-a958cffb

    This sample demonstrates how to create an Entity Framework 5.0 (EF) parameterized query provider to dramatically improve query performance. The sample code shows how to implement IQueryProvider to wrap IQueryable and write a LINQ ExpressionVisitor to translate the ConstantExpression to PropertyExpression before passing the expression tree to EF. The Entity Framework will then compile the expression tree once which greatly improves performance.

    Friday, August 30, 2013 4:07 PM