locked
Non-linear Insert performance on AutoDetectChangesEnabled=true

    Question

  • If we create a DbContext with AutoDetectChangesEnabled=true the time for adding an object seems to grow non-linear with the number of inserted objects.

    The Data class:

    public class Data
    {
      public int Id { get; set; }
      public string Text { get; set; }
    }
    

    The DbContext:

    public class Context : DbContext
    {
      public Context(string connectionString)
       : base(connectionString)
      {}
      public DbSet<Data> Data { get; set; }
      protected override void OnModelCreating(System.Data.Entity.ModelConfiguration.ModelBuilder modelBuilder)
      {
        base.OnModelCreating(modelBuilder);
      }
    }
    

    The "code":

    using (Context context = new Context("Data Source=(local);Initial Catalog=PerfTest;Integrated Security=True"))
    {
      Stopwatch stopWatch = new Stopwatch();
      stopWatch.Start();
      for (int i = 0; i < 10000; i++)
      {
       context.Data.Add(new Data { Text = DateTime.Now.ToString() });
      }
      stopWatch.Stop();
      Console.WriteLine(stopWatch.Elapsed.ToString());
    }
    

    Adding 1.000 Data objects takes 0.5 seconds, 10.000 Data objects takes 30 Seconds (with 100% CPU).

    If we set

    context.Configuration.AutoDetectChangesEnabled = false;
    

    the time consumption is much lower (0.2 seconds for 10.000 objects) and linear (20 seconds for one 1.000.000 objects).

    Is there any additional information about "AutoDetectChangesEnabled" available?

    Thanks and happy new year

     

    Matthias

     

    Monday, January 3, 2011 4:23 PM

Answers

  • Zeeshan: We currently don’t do an optimization inside DetectChanges to determine whether or not we need to scan entities for changes.  The scanning is fast if all entities are change tracking proxies (and don’t have complex types) but it’s still O(n) as opposed to O(1) which it could be with the appropriate optimization in the context.  We will add this optimization to EF but it needs to go in the core assembly so it won’t be available in the first release of Code First/DbContext.

     

    One of the reasons we didn’t do this optimization in the past is that if you have entities with complex types then you may still need DetectChanges.  This is because complex types are mutable but do not provide change tracking notifications.  You can choose to use your complex types as immutable value types (in the DDD sense) in which case you don’t need DetectChanges.  However, this is something you have to decide to do and we can’t tell if you are doing it.  You can switch of AuotDetectChanges in this case and everything should work—but it is the app developer’s decision.

     

    Matthias: My advice would be to use DbContext and drop down to ObjectContext if there are places that you can’t do things that you need with DbContext.  When you do need to drop down to ObjectContext, please let us know so that we can look at whether or not we can make changes to DbContext to cover the case.  We intend DbContext to cover everything that the vast majority of apps would need to do, although we might not get there with the first release and hence dropping down to ObjectContext may be required.  The behavior of DbContext is not intended to be less predictable or provide less control.  Differences from ObjectContext are more things that we feel we didn’t quite get right in ObjectContext and have an opportunity to change.  DbContext is also intended to be much easier to work with than ObjectContext and its related classes.

     

    Thanks,

    Arthur

    Wednesday, January 5, 2011 5:52 PM
    Moderator

All replies

  • Matthias,

     

    Thanks for that very good question.  DetectChanges/AutoDetectChanges is something we should document more clearly.

     

    There is some history behind this.  In EF4 with ObjectContext we only called DetectChanges for you as part of SaveChanges.  The reason for this was precisely because of the potential perf impact that you are seeing.  However, the downside of this is that it is very easy to introduce subtle bugs into your application and then have a hard time tracking them down without a fairly deep knowledge of how EF works internally.  Also, many applications don’t work with large numbers of entities in a single context—although some obviously do—and so never hit these perf scenarios.  Therefore, with DbContext, we opted to call DetectChanges for you in most of the places that you call into the framework.  This was a hard decision but we believe it will help people avoid hard-to-find bugs.

     

    When you need to manipulate many entities (as in your example) the advice is to switch off AutoDetectChanges while you, for example, do all your Adds, and then switch it back on again afterwards. Alternatively, you can choose to always have AutoDetectChanges off and call it yourself when needed.  I would not recommend this unless you are sure you know when you need to call DetectChanges, but it does get you back to a behavior that is more like EF4 with ObjectContext.

     

    Hope this helps.

     

    Thanks,

    Arthur

     

    Tuesday, January 4, 2011 1:01 AM
    Moderator
  • Hi Arthur,

    I created a proxy object usng ctx.Customers.Create<Customer>(); and my class met all the requirements for notification based proxies and a notification based proxy did get created. However the time for inserting 4000 customers was not less. Why would there be a need to call DetectChanges on a notification based proxy which implements IEntityWithChangeTracker. these are the numbers i got.

     

    AutoDetectChanges  = true

    AutoDetectChanges  = false

    notification proxy

    notification proxy & AutoDetectChanges  = false

                                                    9.999

    0.439

    4.806

    0.4881

    9.8445

    0.44

    4.801

    0.4886

    10.094

    0.432

    4.808

    0.483

    9.796

    0.446

    4.91

    0.487

    10.153

    0.441

    4.781

    0.485

    10.153

    0.441

    4.791

    0.488

    9.762

    0.448

    4.771

    0.481

    9.833

    0.443

    4.786

    0.491

    9.8445

    0.447

    4.895

    0.483

                                                    9.942

                                                  0.442

                              4.817

                                                                                              0.486

     

    var ctx = new NorthwindEFEntities();

               // ctx.Configuration.AutoDetectChangesEnabled = false;

                Stopwatch stopWatch = new Stopwatch();

                stopWatch.Start();

                for (int i = 0; i < 4000; i++)

                {

                    var tp = i.ToString();

                    var cust = ctx.Customers.Create<Customer>();

                    cust.CompanyName = tp;

                    cust.ContactTitle = tp;

                    cust.ContactName = tp;

                    cust.Address = tp;

                    cust.City = tp;

                    cust.Region = tp;

                    cust.PostalCode = tp;

                    cust.Country = tp;

                    cust.Phone = tp;

                    cust.Fax = tp;

                    ctx.Customers.Add(cust);

                    //ctx.Customers.Add(new Customer { CompanyName = tp, ContactName = tp, ContactTitle = tp, Address = tp, City = tp, Region = tp, PostalCode = tp, Country = tp, Phone = tp, Fax = tp });

                }

                stopWatch.Stop();

                Console.WriteLine(stopWatch.Elapsed.ToString());

     

     


    Zeeshan Hirani Entity Framework 4.0 Recipes by Apress
    http://weblogs.asp.net/zeeshanhirani
    Tuesday, January 4, 2011 5:41 AM
  • Hello Arthur,

    thanks for this clarifying answer. One additional question is still open:

    Until CTP4 we've made our evaluations based on ObjectContext derived classes. With CTP5 we've made first experiments with DbContext.

    With a ObjectContext derived class like:

      public class ClassicContext : ObjectContext
      {
      private ObjectSet<Data> data;
    
      public ClassicContext(EntityConnection connection)
       : base(connection)
      {
      }
    
      public ObjectSet<Data> Data
      {
       get 
       {
        if (this.data == null)
        {
         this.data = base.CreateObjectSet<Data>();
        }
        return data;
       }
      }
    
      public static ClassicContext BuildContext(DbConnection connection)
      {
       ModelBuilder builder = new ModelBuilder();
       builder.Entity<Data>();
       DbDatabaseMapping mapping = builder.Build(connection);
       DbModel model = new DbModel(mapping);
       return model.CreateObjectContext<ClassicContext>(connection);
      }
     }
    
    the performance is the same as with DbContext and AutoDetectChangesEnabled=false. But in the first case the classic ObjectContext object tracking for object updates is active. The question is now: There are any recommendations when to use DbContext and when to use ObjectContext ? Do we have more control and more predictable behaviour with ObjectContext?
    Tuesday, January 4, 2011 4:14 PM
  • Zeeshan: We currently don’t do an optimization inside DetectChanges to determine whether or not we need to scan entities for changes.  The scanning is fast if all entities are change tracking proxies (and don’t have complex types) but it’s still O(n) as opposed to O(1) which it could be with the appropriate optimization in the context.  We will add this optimization to EF but it needs to go in the core assembly so it won’t be available in the first release of Code First/DbContext.

     

    One of the reasons we didn’t do this optimization in the past is that if you have entities with complex types then you may still need DetectChanges.  This is because complex types are mutable but do not provide change tracking notifications.  You can choose to use your complex types as immutable value types (in the DDD sense) in which case you don’t need DetectChanges.  However, this is something you have to decide to do and we can’t tell if you are doing it.  You can switch of AuotDetectChanges in this case and everything should work—but it is the app developer’s decision.

     

    Matthias: My advice would be to use DbContext and drop down to ObjectContext if there are places that you can’t do things that you need with DbContext.  When you do need to drop down to ObjectContext, please let us know so that we can look at whether or not we can make changes to DbContext to cover the case.  We intend DbContext to cover everything that the vast majority of apps would need to do, although we might not get there with the first release and hence dropping down to ObjectContext may be required.  The behavior of DbContext is not intended to be less predictable or provide less control.  Differences from ObjectContext are more things that we feel we didn’t quite get right in ObjectContext and have an opportunity to change.  DbContext is also intended to be much easier to work with than ObjectContext and its related classes.

     

    Thanks,

    Arthur

    Wednesday, January 5, 2011 5:52 PM
    Moderator
  • I don't quite understand the part u mentioned about complex type. From what i know regardless what property u change on the complex type, EF would always create an update to update all columns for a complex type. if i understand correctly then the following is true.

    customer.Address.Address1 = "changed adderss";

    db.DetectChanges();

    customer.EntityState == UnChanged

    compared to

    customer.Address = new Address{Address1 = changed address"};

    db.DetectChanges();

    customer.EntityState = EntityState.Modified

    Also in my model the only entity i had was customer which which had change tracking proxy and still the time it took was 4.8se compared to .42 sec. 


    Zeeshan Hirani Entity Framework 4.0 Recipes by Apress
    http://weblogs.asp.net/zeeshanhirani
    Wednesday, January 5, 2011 6:26 PM
  • Zeeshan,

     

    If you do this:

     

    customer.Address.Address1 = "changed adderss";

     

    where “customer” is a change-tracking proxy and “Address” is a complex property, then Address will not be marked as Modified until DetectChanges is called.  This is because the complex type itself is not a proxy and does not report changes.  There are very significant hurdles to making the complex type a proxy and having it report changes so we opted not to do so, especially since if you treat your complex types as immutable then you don't run into this problem.

     

    On the other hand, if you do this:

     

    customer.Address = new Address{Address1 = changed address"};

     

    then Address will be marked as Modified immediately because you changed a property on the change-tracking proxy and therefore the change was reported immediately.

     

    The times are as you report because we have not implemented the optimization to perform no action if all types are change tracking proxies with no complex types.

     

    Thanks,

    Arthur

    Wednesday, January 5, 2011 6:47 PM
    Moderator
  • Hello Arthur,

    thanks for your advice. This forum is a great help for us!

    Friday, January 7, 2011 7:33 AM