none
EF 4.0 + DDD + DTO => troubles with persistance of complex object graphs

    Question

  • Hello,

    this is not the question about EF 4.0 functionality but about architecting application with EF 4.0 generally.

    I'm working on Proof of Concept for using EF 4.0 and other .NET 4.0 features in complex architecture:

    • I have repositories to deal with persistance - EF 4.0 based.
    • I have unit of work which internally shares context among repositories
    • I have set of POCO classes. Those classes represents domain entities so they also include business methods. Some properties can be modified only by calling business methods. Repositories persists those classes and returns them as results to queries.
    • I have service layer (not services from DDD). It is facade layer which exposes business layer functionality. Internally it uses repositories and domain objects. Domain objects are not exposed - each service method uses set of specialized Data transfer objects for parameter and return value. It is responsibility of each service method to transform domain entity to DTO and vice-versa.
    • I have ASP.NET MVC application which uses service layer and DTOs.

    This architecture should also prove ability to scale out the logic from single tier to two tiers without touching existing layers. This should be possible by adding two layers between service layer and ASP.NET MVC and configure IoC container.

    • WCF service wrapper on the top of the service layer.
    • WCF proxy layer for client exposing same interface as the service layer.

    The problem of this architecture is tracking changes in complex entities (aggregate roots). I'm not using dynamic tracking proxy or self-tracking entities because they generally don't help me with the task and make my POCO classes ugly.

    Suppose simple example:

    public class Article
    {
     public int Id { get; set; }
     public string Title { get; set; }
     public string Body { get; set; }
     public byte[] Timestamp { get; set; }
    
     private ICollection<Keyword> _keywords;
     // M:N mapping
     public ICollection<Keyword> Keywords
     {
      get
      {
       return _keywords;
      }
     }
    
     public Article()
     {
      _keywords = new List<Keywords>();
     }
    }
    

    This example contains Article entity which contains collection of related Keyword entities. Now suppose that I want to modify article.

    1. Browser calls web server to get article for edit.
    2. APS.NET MVC application calls the service layer to get the article for edit.
    3. Service call repository gets the domain object. Than the object is converted to DTO and returned to the ASP.NET MVC. Original domain object is lost.
    4. ASP.NET MVC application gets DTO and renders the page for article editation. Original DTO is lost.
    5. Artilce is modified in the browser. This can take some time and even include some asynchronnous request to ASP.NET MVC. The result is the article which can have modified content and modified set of keywords. Some keywords could be removed some keywords could be added and even some new (not yet existing in the application) could be submitted.
    6. Browser submits new version of the article to the web server.
    7. ASP.NET MVC binds the request to the new instance of DTO and passes the DTO to the service layer operation for saving article.
    8. Service layer gets DTO and converts the DTO to domain object. It than calls some domain object's methods and persists the domain object using repository.

    The issue here is between steps 3. - 8. This approach don't use any change tracking so it works only for simple entities without modification of relations and related entities. I have studied some approaches to deal with a problem but I got stuck.

    I can add some very simple change tracking into my DTOs to track information about relations and modifications of related entitis (I don't need to track modification of each property at the moment). This tracking should to be very simple and mostly manual (no logic in properties). Than I can persist some information about intial state into rendered page. This will allow me to track changes in ASP.NET MVC part. Note: I don't want to use Session to store original DTO. But than the problem is in step 8. I have the DTO and I know what has changed. But how to transfer this information from DTO to domain entity so that it is correctly persisted? I see several approaches but no one seems good enough to me:

    • Use dynamic proxy tracking entities (POCO mapped members marked as virtual). But this requires that before each update the current state is loaded from database and DTO is updated into loaded domain object. It means additional query to database (which can be complex), slow dynamic proxy and another level of concurrency issues. Similar problem is with self tracking entities which are faster than dynamic proxies but other problems are the same. Also the self tracking entity is not my POCO. Main reason for not accepting this solution is additional query to get current state of the object - or is this common in DDD?
    • Pass DTO to repository and do conversion to Domain object there, than use ObjectStateManager to properly set EntityState for each object and relationship in an object graph based on tracking information stored in the DTO. Repository and Unit of Work are only classes aware of the EF so using ObjectStateManager elsewhere is not possible. This approach is absolutely inacceptable. Repository has to work with domain object. Moreover DTO is from service layer it can't be used in underlaying layers.  
    • Add some information into domain object so that change tracking from DTO is passed to the repository. Use ObjectStateManager to properly set EntityState for each object and relationship in an object graph based on tracking information stored in the domain object. This looks like the only usable approach but it requires modification of POCO. Also my first attempts were not very successful. I had problems with deleting relationships and adding relationships to newly created entities.  

    I assume this has to be a problem which many developers who use any ORM tool had to solve before. I will be happy if you share some your experience or propose another solution. 

    Thanks in advance for help.

    Best regards,
    Ladislav

    Sunday, August 01, 2010 9:44 PM

Answers

  • Hello,

    there is only one possible solution - manual change tracking. My solution received pure DTO with new object graph, I loaded current graph of entites from new ObjectContext (new query to DB) using virtual tracking proxies and I merged data from DTO into loaded object graph.

    I have added some theory about this problem to the question on StackOverflow.

    Best regards,
    Ladislav

    • Marked as answer by Roahn Luo Monday, September 06, 2010 2:03 AM
    Saturday, September 04, 2010 2:08 PM

All replies

  • On 8/1/2010 5:44 PM, Ladislav Mrnka wrote:
    > Hello,
    >
    > this is not the question about EF 4.0 functionality but about
    > architecting application with EF 4.0 generally.
    >
    >
    > I assume this has to be a problem which many developers who use any ORM
    > tool had to solve before. I will be happy if you share some your
    > experience or propose another solution.
    >
    > Thanks in advance for help.
    >
     
    I used EF 3.5 in 3 major n-tier applications.
     
    UI
    MVP
    Service Layer
    WCF Web service
    BLL
    DAL
    model
     
    DTO(s) were used that kept their state of isNew, isDirty, IsDelete and
    isValid.
     
    The parent DTO had related DTO collections within it.
     
    It made it kind of simple to persist DTO(s) that were mapped to entities
    in a relationship and persist them and the parent, because you could use
    Linq to query a collection for insert, update and delete DTO(s) and map
    them to entities along with taking the appropriate path in code to
    persist them and the parent.
     
    Sometimes simpler is better.
     
     
     
     
     
     
    Monday, August 02, 2010 12:17 AM
  • Hello Darnold,

    thanks for response. That is the part I understand.  I can track changes in DTO in similar way. The problem here is converting DTO to Entities. In your case you have EF 3.5 so I suppose you query the database for current state of entity (or load it from some cache) and get entity object attached to context. Than you transfer data from DTO to Entities - context track changes for attached entity so it can be easily persisted. I have two problems with this approach:

    • Additional query is needed to get entity in state before modification.
    • Entity has to be able to track its changes which is not a case for pure POCO.

    Best regards,
    Ladislav

    Monday, August 02, 2010 5:57 AM
  • Hi,

    Could you please share with us which solution did you choose? I have exactly the same problem (even with Articles and Keywords :) ).

    Thank you.

    Friday, September 03, 2010 2:55 PM
  • Hello,

    there is only one possible solution - manual change tracking. My solution received pure DTO with new object graph, I loaded current graph of entites from new ObjectContext (new query to DB) using virtual tracking proxies and I merged data from DTO into loaded object graph.

    I have added some theory about this problem to the question on StackOverflow.

    Best regards,
    Ladislav

    • Marked as answer by Roahn Luo Monday, September 06, 2010 2:03 AM
    Saturday, September 04, 2010 2:08 PM
  • Ladislav,

    Thanks for your extensive posting on the subject.  This issue has proved to be a major obstacle in adopting EF in our shop.  Would you be willing to post some sample code?  I have read several of your posts on StackOverflow, but am confused about exactly how to implement the solution you are suggesting.  Some basic code snippets would be greatly appreciated!

     

    Thanks, DanO

    Thursday, April 28, 2011 8:08 PM
  • On 4/28/2011 4:08 PM, danielcobrien wrote:
    > Ladislav,
    >
    > Thanks for your extensive posting on the subject. This issue has proved
    > to be a major obstacle in adopting EF in our shop. Would you be willing
    > to post some sample code? I have read several of your posts on
    > StackOverflow, but am confused about exactly how to implement the
    > solution you are suggesting. Some basic code snippets would be greatly
    > appreciated!
     
    Using DTO(s) and mapping data between DTO and EF entity and persisting
    the entity to a database is a simple thing to do even using a complex
    object graph that contains List <T> of objects within an object. Are you
    are using DTO(s) passing the DTO through layers?
     
    Thursday, April 28, 2011 8:18 PM
  • Hello Daniel,

    it is completely manual process and it whole depends on the actual problem you are facing. Much better then posting some code to demonstrate solution for particular problem is working with the real problem you are facing. Post a question either here or on Stack Overflow and I will check if I'm able to describe the solution during weekend.

    Best regards,
    Ladislav

     

    Thursday, April 28, 2011 10:24 PM