Swappable data access layers - has anyone any case studies or evidence? RRS feed

  • Question

  • User-1611549905 posted

    I've been quite vocal (both here and on my blog) that attempting to abstract out your data access layer so that you can swap out NHibernate for Entity Framework, or Entity Framework for a different data access technology (e.g. RavenDB, cloud data hosting etc) is a waste of time and effort.

    The basis for this opinion is partly my own experience -- I've seen many projects get bogged down with anaemic business layers, restrictive or bloated repositories, and unnecessary copying of data from entities to identical DTOs, which has just made it harder to make common changes such as adding a column or a many-to-many relationship, and, far more seriously, introduced performance problems that in some cases have been quite severe (select n+1 issues resulting in tasks that should take a minute at most taking several hours if in fact they don't time out altogether). On the other hand, the need to swap out one ORM or data access technology for another seems to be pretty rare.

    There are also a couple of blog posts that I've cited that are quite informative:

    • "The false myth of encapsulating data access in the DAL" by Oren Eini aka Ayende Rahien (former core contributor to NHibernate; lead developer of NHibernate Profiler, Entity Framework Profiler, Rhino Mocks, RavenDB; CEO of Hibernating Rhinos) gives a good overview of the technical reasons why trying to abstract your DAL to make it swappable doesn't work in practice.
    • "Migrating from NHibernate to Entity Framework" by Jimmy Bogard (lead developer of AutoMapper) gives a case study of a real-world attempt to switch from NHibernate to Entity Framework. He reported that making the switch only took about three days altogether for a project with about 60-70 entities that had not made any special considerations for the need to swap data sources. Switching from the ISession querying/updating API to the DbContext API was largely an exercise in regular expressions; the bulk of the work involved entangling fundamental and thorny differences where an abstraction layer would not have helped (e.g. deep-seated differences in the way in which one-to-one mappings are supported).

    The conclusion that I've drawn from all of this is that trying to abstract out your DAL so that you can swap out one data source for another doesn't save you anything in terms of effort, but just adds a whole lot of complexity.

    I have had a couple of people push back on me with this though, asserting that switching ORMs or data sources is necessary from time to time. I can see that this might occasionally be the case, but nobody has presented me with any concrete evidence or case studies to back up the assertion that abstracting out your DAL makes enough of a difference in practice to justify the extra overhead that it creates in terms of additional effort and risk.

    What I'd like to see here are some case studies from people who have actually had to swap out one ORM or data access strategy for another in practice. I'm not talking about people who did it just because they wanted to, but because there was a genuine business need. In particular:

    • Roughly what percentage of projects do you find actually require your data access strategy to be switched out for another one in practice?
    • What business reasons were you given for having to swap one out for another?
    • How long did it take in the end?
    • What particular problems did you encounter when you did so?
    • If you had attempted to plan for this contingency (for example, by abstracting out your DAL), did this actually make a difference in practice? Roughly how much longer do you think it would have taken without such an abstraction layer, and roughly how much work was involved that your abstraction layer did not address?
    • Do you think on hindsight that you could have organised your abstraction layer differently to make it easier? If so, how?
    • Do you have any specific evidence to counter the points made by Oren Eini and Jimmy Bogard in the blog posts I linked to?
    • Did swapping out your database/data access technology actually solve anything in practice?

    Another, similar one that I've encountered has been claims that you might need to move layers onto separate tiers. Again, has anyone any real-world experience of actually having done this and did it actually solve the problem concerned?

    I'd just like to reiterate here that I'm not looking for people saying "you should do this" or "this is a best practice." I'm looking for evidence.

    Tuesday, September 9, 2014 5:00 AM

All replies

  • User-760709272 posted

    You keep going on about select n+1 issues....are you aware that using an ORM like EF is *more* likely to see you have select n+1 issues as EF lazy-loads by default?  You say ORMs are superior to repo patterns as you can leverage lazy loading....which gives you select n+1 problems.  Which one is it?  Either you don't know what select n+1 is, or you don't know how to properly write a repo and are blaming the pattern for your poor knowledge of implementation.

    You complain about bloated repositories....so EF *isn't* bloated?  Which one is it?  Bloated repos are bad but bloated ORMs are good?

    As I've said before, you are incredibly inconsistent with your thought processes which makes it very hard for me to take you seriously.

    You then talk about tasks that should take a minute at most....if a minute is acceptable to you, I don't want to use any website you've written :)  If I had a task that took more than 100ms I'd be looking to improve it.  You then claim that using a repro can result in this task taking several hours instead.  Are you for real?  Again I come back to an earlier point, I fear your distaste of patterns is down to you not implementing them properly.  Or...worse, your claim that tasks that will take a minute take several hours is just utter nonsense that you've made up.

    I could give you real world evidence of the benefits of data abstraction, I could give you evidence of the rigid architecture you recommend resulting is massive re-writes for simple changes, and I could give you evidence of data abstraction not helping....however the first you will dismiss as not true\not valid, the second you will blame on something other than the architecture, and the last you'll accept as truth without question.  To be frank, I think you started this thread just so you could employ fallacious arguments, ignore the benefits of what anyone else recommends, and the ignore the flaws of what you recommend, purely to continue an argument that no-one really wants to have - the majority are just here to use our experience to advise others in a (hopefully) unbiased way. So you hold on tight to those two blogs you repeatedly post, while the rest of us continue to reap the benefits of flexible, loosely-coupled architectures :)

    Tuesday, September 9, 2014 4:58 PM
  • User-1611549905 posted

    @AidyF: You seem to be persistently misunderstanding me to the extent that I question whether you are even reading my posts correctly. In particular, you seem to be repeatedly claiming that I am saying things that I am not, and treating specific criticisms that I have made about specific patterns and practices applied in specific situations as if they were across-the-board generalisations in ways that I never intended. I wonder if that's why you are finding yourself a bit confused by some of the things I'm saying?

    I made this post because I am asking for something very specific that our previous discussions have not addressed: whether anyone had any real-world experience of dealing with the requirements that the specific separation between your layers in a traditional BLL/BOL/DAL architecture is supposed to facilitate, and could provide some evidence in the form of case studies, retrospectives, lessons learned, and so on. The only reports that I've seen so far have been to the effect that it fails to deliver; I was rather hoping that somebody (such as yourself) might be able to provide evidence for a contrary viewpoint if it really is as beneficial as you claim it is.

    I could answer your other points in more detail if you like, but I would prefer to get this thread back on topic, so it would be better to conduct such a discussion on a different forum if necessary. If you have something constructive to bring to the table here in terms of evidence, experience reports, case studies and answers to my questions, then by all means do, but please don't try to derail the discussion with ad hominem and straw man arguments.

    Wednesday, September 10, 2014 4:53 AM
  • User-760709272 posted

    I could give you plenty of evidence, but from your past posts doing so would be useless as you'd merely dismiss everything put to you that doesn't agree with your own opinions *shrug*

    Wednesday, September 10, 2014 5:24 AM
  • User-1611549905 posted

    Exactly which of my past posts leads you to draw that conclusion and why?

    Wednesday, September 10, 2014 5:27 AM