none
Feature Auto-Compiled LINQ Queries for Code First RRS feed

  • Question

  • This cost of query compilation appears to be causing a problem for a query with a where and group by clause in an Azure web project with a under 1000 rows and it is not workable client solution.  Is there an estimate delivery date for this feature?

    Thanks.

    Wednesday, October 5, 2011 4:32 PM

Answers

  • Hello Ben,

    I agree in general that the optimal combination of server and client processing will depend on several factors, such as the schema, the database engine capabilities, the machine and network configuration, etc.

    If you have tested this and it provides better performance than the server-side queries you had before, I think it makes sense to use it. There are a few things I would still consider:

    1. You probably already tried this but I would play with lifting the AsEnumerable() a bit, i.e. you can do it just before the OrderBy or you can consider changing the projection to obtain Bids and Offers collections from the server and then do FirstOrDefault on the client in a separate projection.

    2. You mentioned that query compilation seemed to be the issue, but is there any possibility that query processing on the server could also have an impact? Does the database contain adequate indexes to resolve this query?

    3. For a web application server adding parallelism might show you a performance improvement but cause scalability issues, as spinning up additional threads will increase the memory footprint. With enough concurrent requests you might see that the memory of the server is exhausted before the processor reaches a high utilization. I am not familiar enough with parallel LINQ to know whether it does anything to help avoid this.

    Hope this helps,
    Diego


    This posting is provided "AS IS" with no warranties, and confers no rights.
    Tuesday, October 11, 2011 1:55 AM
    Moderator

All replies

  • Hi benaw,

    You can refer here: http://blogs.msdn.com/b/efdesign/archive/2011/06/30/auto-compiled-linq-queries-entity-framework-june-2011-ctp.aspx 

    and

    http://thedatafarm.com/blog/data-access/entity-framework-amp-wcf-data-services-june-2011-ctp-auto-compiled-linq-queries/

    Have a nice day.


    Alan Chen[MSFT]
    MSDN Community Support | Feedback to us
    Get or Request Code Sample from Microsoft
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Friday, October 7, 2011 3:48 AM
    Moderator
  • Hello Benaw,

    To add to the information provided by Alan: the current plan is that auto-compilation of LINQ queries, alongside other features that were originally previewed in the June 2011 CTP of EF, will be part of the update of the EF core libraries we will ship with .NET Framework version 4.5. At the moment we don't have a schedule we can share for the .NET 4.5 release.

    In the meanwhile I would suggest trying to convert this particular query to a raw SQL query and use SqlQuery to materialize the results as a workaround.

    Hope this helps,

    Diego


    This posting is provided "AS IS" with no warranties, and confers no rights.
    Friday, October 7, 2011 10:31 AM
    Moderator
  • Ok, thanks for this.

    This option has occurred to me and I have been exploring these other options.  One which, is worth sharing is to reduce the server query size, bring back more data but perform more on the client (web role) side, which suprising has had over 100% improvement in performance by just a couple of statement changes in bold (for me on this particular set of data :) ) but obviously look forward to the CTP going live.

    var results = db.Prices.Where(price => ...)
                           .GroupBy(price => new { price.Product, price.Amount }).AsEnumberable()
                           .Select(g => new Spread
                           {
                                Product = g.Key.Product,
                                Amount = g.Key.Amount,
                                Bid = g.Where(p =>  ... && p.Date == g.Where(x => ...).Max(y => y.Date)).FirstOrDefault(),
                                Offer = g.Where(p => ... && p.Date == g.Where(x => ...).Max(y => y.Date)).FirstOrDefault()
                            }).AsParallel()
                              .OrderBy(...)

    Regards Ben. (very loosely inspired by map reduce)  Any thoughts let me know.


    • Edited by benaw Friday, October 7, 2011 12:12 PM code
    Friday, October 7, 2011 11:59 AM
  • Hello Ben,

    I agree in general that the optimal combination of server and client processing will depend on several factors, such as the schema, the database engine capabilities, the machine and network configuration, etc.

    If you have tested this and it provides better performance than the server-side queries you had before, I think it makes sense to use it. There are a few things I would still consider:

    1. You probably already tried this but I would play with lifting the AsEnumerable() a bit, i.e. you can do it just before the OrderBy or you can consider changing the projection to obtain Bids and Offers collections from the server and then do FirstOrDefault on the client in a separate projection.

    2. You mentioned that query compilation seemed to be the issue, but is there any possibility that query processing on the server could also have an impact? Does the database contain adequate indexes to resolve this query?

    3. For a web application server adding parallelism might show you a performance improvement but cause scalability issues, as spinning up additional threads will increase the memory footprint. With enough concurrent requests you might see that the memory of the server is exhausted before the processor reaches a high utilization. I am not familiar enough with parallel LINQ to know whether it does anything to help avoid this.

    Hope this helps,
    Diego


    This posting is provided "AS IS" with no warranties, and confers no rights.
    Tuesday, October 11, 2011 1:55 AM
    Moderator