none
WCF Data Services And Repository Pattern

    Question

  • Hi,.

    • I am at the beginning point of designing an SOA. The SOA consists of 'n' services, which should be able to exchange data, as well as send/receive operation calls, with each other.
    • In addition, I would like to use a domain-driven design approach to accessing and handling data (i.e. ultimately some sort of automatic object-mapping approach) so that I don't have to worry too much about how my domain objects (or entities) are stored.
    • This is partly because I want to decouple from the underlying storage tech (e.g. SQL RDBMS, MongoDB document database etc) and partly because for client apps talking to the service(s), I would like to use MVVM design pattern - and having POCO entities lends itself to this design patttern.
    • To decouple from the underlying data tech, I want to use repository pattern/unit of work pattern.

    Option 1 - Custom WCF Service

    • One option to implement the above architecture would be a custom data and operations service, tailored to my particular application - e.g. using WCF's default SOAP serialization to send/receive data and operations contracts.
    • The service could expose my repository interfaces (e.g. add/remove/create new entities etc. via the standard CRUD operations you would support).
    • Any external client would need a service API DLL from me per service, which would provide a proxy connection to the service and which they could use to query the service for data, or request it to perform operations.
    • But this means every time I change my data schema, or supported operations, I need to issue a new DLL version and any new clients need to re-implement using this DLL.

    Option 2 - WCF Data Services

    Another option would be WCF Data Services (which is Microsoft's implementation of OData protocol).

    1. With OData, HTTP requests are such that you don't need to create specific requests when you add new entities to your data schema (i.e. I don't have to implement or handle specific new calls when I change the schema) - because these are handled in the parameter syntax of the OData service calls.
    2. Implementation details such as filtering, paging, transactions etc. are built into the OData architecture already.
    3. Any external clients could use pre-made OData client APIs to access the service - no need to get an updated DLL from me when the data schema changes or new operations are added (is this correct?)
    4. Interfaces such as IUpdatable allow you build your own WCF Data Services context, so in theory, we can use the service to access our data via our Repository/Unit Of Work architecture rather than being tied to a particular shipped data context (e.g. the Entity Framework context) and hence an underlying data access technology (which is not what I want).

    This seems like the dream - a well-defined, highly supported data API accessing a fully decoupled data store. And if I add new operations, or change the data schema, I don't have to issue a new DLL to all users of the service (at least I think this is the case).

    The Problem - Finding A Good Implementation Of IUpdatable

    Only trouble is...

    I cannot find a lot of examples of people successfully using repository pattern with a WCF data service; whether ultimately you end up with unforeseen performance problems, and the books I have looked at on WCF Data Services seem to focus on the pre-shipped implementations for Microsoft SQL Entity Framework.

    I don't want to use the pre-shipped context, I want a context that accesses my data via my repository by implementing IUpdatable (http://msdn.microsoft.com/en-us/library/system.data.services.iupdatable(v=vs.113).aspx)

    What I Need

    • Can anyone indicate concrete examples of this architecture being implemented, or any literature that helps with the implementation of IUpdatable that would be required to access a generic repository?
    • I imagine entities would e.g. use IEquatable<> to indicate the ID propert(ies), and perhaps support INotifyPropertyChanged as well.
    • Is the use of repository pattern with WCF Data Services unnecessary, given the context you pass the WCF Data Service effectively decouples you from the underlying data store anyway?

    Thanks for any thoughts. I hope I haven't repeated a question, as I say I have only seen specific questions on this rather than the general design approach.

    Thanks

    Markos101



    Monday, November 18, 2013 8:43 AM

Answers

  • Hi,

    >>Does anyone have any thoughts on change notification with OData Services - whether it is necessary and if there are any implementations out there already that incorporate 'push' change notification in some way?

    Unfortunately, there is no functionality in Data Services that would do this kind of notification for you - if this is a hard requirement for you, you would need to implement something to do this for you. If you think about what Data Services is (an HTTP REST based service) this type of notification from the server to the client would violate the spirit of a REST based service. It would require the service to maintain significant amounts of cross-request state that HTTP servers typically don't do.

    There is something called WCF callbacks. Take a look at the example here.

    http://www.codeproject.com/Tips/348389/Basic-WCF-Call-plus-Callback-example

    Perhaps this is something that you can use to solve this issue.

    Regards.


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Monday, November 25, 2013 9:42 AM

All replies

  • Hi,

    I will move your question to WCF Data Service Forums to get more help.

    Thanks for your understanding.

    Best Regards,
    Amy Peng


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.


    Tuesday, November 19, 2013 5:39 AM
  • Hello,

    >> I don't want to use the pre-shipped context, I want a context that accesses my data via my repository by implementing IUpdatable

    (http://msdn.microsoft.com/en-us/library/system.data.services.iupdatable(v=vs.113).aspx)

    Being strange that why you want to use repository. Since we have context so use it and WCF data service already handles everything you need itself and it sometimes offers even more (for example interceptors).

    By using custom repository, please have a look at link below:

    http://msdn.microsoft.com/en-us/library/dd728281.aspx?cs-save-lang=1&cs-lang=csharp#code-snippet-1

    And since you also plan to modify your entities through WCF data service that is also against your repository because reflection provider is read only unless it also implements IUpdateable. So we have to define ourselves provider, for this, please check Custom Data Service Providers written by Alex.

    Regards.


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Tuesday, November 19, 2013 8:24 AM
  • Hi - thanks for your response and for the helpful link.

    I understand that the DataContext provides an interfaced implementation of access to the underlying data technology, and so in theory I could switch my data context for one which supported another data storage technology; et voila, I am decoupled.

    However, I have a number of primary concerns about this approach. Could you comment on these?

    • Tie to a particular data access technology. Let's assume at some point in the future MS ditch SQL or Entity Framework for something bigger and better. In reality, how easy is it in a practical implementation of a WCF data service to rely on simply changing the data context I use to change my underlying data access technology? Can you rely on simply swapping the data context to another one later without lots of refactoring?
    • Unit testing What if I want to write some parts of my data to memory for unit testing? I can't use the Visual Studio unit testing framework if I need a database running - it breaks the cardinal rule of unit testing.
    • All my data has to be managed by a single data access technology. What if I want part of my entity model exposed by my WCF data service to only go to memory, rather than an Entity Framework database, but then other parts to be written to a database? Do I have to expose two services; one for writing to a database, another for writing to memory only? By decoupling from the data access technology at the provider level, I have this flexibility.

    Markos101


    • Edited by Markos101 Tuesday, November 19, 2013 8:51 AM
    Tuesday, November 19, 2013 8:49 AM
  • Hi, still interested in a response to these points, however I have come across this blog post which seems to most closely resemble the approach I am looking at (MVVM with a repository based WCF OData Service) - although I am looking at using WPF rather than Silverlight:

    http://aviadezra.blogspot.co.uk/2010/10/silverlight-mvvm-odata-wcf-data.html

    Are there any other resources available documenting best practices of using a WCF data service with MVVM (optionally using a repository)?


    PS - it is a real shame Silverlight has been rather abandoned by Microsoft to the extent now new projects can't really invest in using it for the future. It is difficult to find as rich a client UI API as XAML, and HTML5 does not really compare. W3C have really let everyone down with this one (credit though for plenty of other good stuff besides!).
    • Edited by Markos101 Tuesday, November 19, 2013 10:50 AM
    Tuesday, November 19, 2013 10:46 AM
  • Hi,

    >>In reality, how easy is it in a practical implementation of a WCF data service to rely on simply changing the data context I use to change my underlying data access technology? Can you rely on simply swapping the data context to another one later without lots of refactoring?

    Do not quite understand it? Does it mean that you want to implement multiple data context for a single wcf data service?

    >>Unit testing

    Although it's probably not a good idea to test a WCF Data Service directly - I'd prefer to test the underlying Entity Framework model and any web methods' logic directly without the OData infrastructure.

    And for Unit Testing ADO.NET Data Services, the blog written by ploeh shows details regarding it.

    >> All my data has to be managed by a single data access technology

    Actually you do not need to worry this, because it has been considered.

    After we create a wcf data service, we will have a method like below:

    public static void InitializeService(DataServiceConfiguration config)
    
            {
    
                // TODO: set rules to indicate which entity sets and service operations are visible, updatable, etc.
    
                // Examples:
    
                config.SetEntitySetAccessRule("*", EntitySetRights.All);
    
                config.SetServiceOperationAccessRule("*", ServiceOperationRights.All);
    
                config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V3;
    
            }
    

    And it accepts a DataServiceConfiguration object as the parameter.

    Now we could use this object to config which part of the entity model should be exposed and decide rights of the exposed entities, read-only, write-only or others.

    For more information regarding the config:

    http://msdn.microsoft.com/en-us/library/ee358710(v=vs.110).aspx

    >> Are there any other resources available documenting best practices of using a WCF data service with MVVM (optionally using a repository)?

    I find the article regarding WPF with WCF data service:

    http://blog.tonysneed.com/2010/04/13/wcf-data-services-versus-wcf-soap-services/

    You can have check it.

    Regards.


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Wednesday, November 20, 2013 8:42 AM
  • Hi Fred, thanks for your responses.

    • ">> Do not quite understand it? Does it mean that you want to implement multiple data context for a single wcf data service?"

    No. I don't want to implement multiple data access technologies for a single WCF data service. But I do want to abstract the classes I use to store and access my data (normally called domain classes) away from the data access technology itself, using repository pattern, so that I can easily swap the data access technology the WCF data service uses.

    The purpose of using repository pattern (http://msdn.microsoft.com/en-us/library/ff649690.aspx) is to make my domain classes agnostic to the data access technology that reads, writes or updates them. To do this, I define interfaces for the model I want to read, write and update - and implement specific classes that use these interfaces directed at a specific data access technology (e.g. Entity Framework).

    This allows me to (a) change the data access technology without effecting the interfaces my business logic sees that describe my domain model, and (b) allows me to unit test easily by making in-memory only implementations of my repository interfaces (instead of reading my data from a database connection, which isn't allowed for the purposes of unit testing).

    Correct me if I am wrong, but if I use an Entity Framework data context for my WCF data service, suddenly my data context expects DbSet<> and other EF specific classes in the interfaces my repository exposes (and that my data access technology specific domain classes implement)? Suddenly, I'm tied again to the specific data access technology and lose the two major advantages (a) and (b) I've defined above - the very two things repository pattern was designed to bring into being.

    So by saying to the WCF data service 'use this Entity Framework data context' - I can no longer use a generic repository that abstracts me away from Entity Framework. Does that make sense? It's a subtle but important point - happy to be corrected on this if I am mistaken.

    • ">> Although it's probably not a good idea to test a WCF Data Service directly - I'd prefer to test the underlying Entity Framework model and any web methods' logic directly without the OData infrastructure."

    That is fine, but still breaks the cardinal rule of unit testing - I shouldn't be testing the Entity Framework model because this requires an SQL database, which by definition isn't unit testing. Removing the OData infrastructure but leaving the requirement to have an Entity Framework database connected to unit test still breaks this rule - I need a database to perform my unit tests. If I can't substitute what I would normally access through Entity Framework with an in-memory model (e.g. using repository pattern) which doesn't require an external database for unit testing purposes, I still can't do unit testing.

    • ">> All my data has to be managed by a single data access technology

    Actually you do not need to worry this, because it has been considered."

    Okay, thanks, I will take a look at the method override you've outlined.

    • ">> I find the article regarding WPF with WCF data service

    http://blog.tonysneed.com/2010/04/13/wcf-data-services-versus-wcf-soap-services/"

    Thanks, I will take a look at this.


    Would be grateful in particular for your response on the first point.
    • Edited by Markos101 Wednesday, November 20, 2013 9:44 PM
    Wednesday, November 20, 2013 9:30 PM
  • Hello Markos101,

    It seems that you can use the WebAPI in MVC.

    The WebAPI also uses OData and use the S/C model and it can let us use repository to define ourselves domain classes.

    For web api and entity framework:

    http://www.asp.net/web-api/overview/creating-web-apis/using-web-api-with-entity-framework/using-web-api-with-entity-framework,-part-1

    Regards.

    Thursday, November 21, 2013 8:50 AM
  • This video explains how to create a WCF data service that accesses a simple static in-memory data store comprised only of POCOs, with no decoration with data access technology specific attributes or anything else.

    All you need to do is expose your top-level collection of POCOs as an IQueryable<> and pass this context to the service.

    http://msdn.microsoft.com/en-us/data/gg601461

    This is useful, in that it clarifies that you can indeed expose a generic repository to a WCF data service. What it doesn't show in this example is how you would implement IUpdatable<> such that you could carry out full CRUD operations on it (in the video it only shows reads).

    I think it's time I really started something to move forward, and with what I end up with I'll post back on here. I'm happy that in theory you can write a generic IUpdatable<> to populate a repository model based only on POCOs.

    A couple of things that still bug me:

    • For WCF data services to be able to read an IEnumerable (wherever that IEnumerable came from) you need to expose it as IQueryable. This allows URLs called on the data service to be translated into LINQ queries underneath so that the results can be returned (for example, a filter query on the POCOs in the IEnumerable).
    • This is nice, however sort of breaks the ideals of repository pattern where you don't expose IQueryable<> - you only expose FindXXX() methods on your repositories as and when you need them.
    • The saving grace here is that almost all standard list or collections in .NET implement IQueryable<>, and it's these collections that you would use in pretty much any repository. So it just so happens that in the .NET case a repository list implementation would almost always support IQueryable<> anyway.
    • You can, however, still define custom FindXXX() methods on your repository POCOs and expose these in your WCF data service. Looks nice (at least at the moment!).

    Anyway, I will update here when I've got my all-singing, all-dancing generic IUpdatable implementation acting on my generic repository.


    • Edited by Markos101 Thursday, November 21, 2013 9:27 AM
    Thursday, November 21, 2013 9:25 AM
  • Just an update from me on this.

    • I've found the 'OData Programming Cookbook for .NET Developers' book useful in going into more detail on how you can customize an OData service.
    • Reading further details on the OData interfaces the framework is actually very flexible indeed, which is good.
    • Normally, you can easily create a read-only memory POCO based entity model for your WCF Data Service to expose by using the Reflection Provider (as has been mentioned already). All you need to do is apply data service attributes to these POCO entities, to indicate to the service e.g. which property it should treat as the Key property for a type.
    • By going further and implementing your own data service provider (e.g. IDataServiceMetadataProvider, IDataServiceQueryProvider, IDataServiceUpdateProvider) interfaces, you can even handle pre-existing entity types (e.g. those provided to you by someone else, that you yourself don't have the ability to apply data service attributes to).

    Service Side

    I think the easiest way to implement the data service without tying my entity model to a particular data access technology (e.g. Entity Framework) would be to, for each data access technology I use, implement my repository interfaces (that just expose IQueryable) for each data access technology.

    If I wanted to use EF for example, I would just make the service data context (derived from DbContext) implement my Unit Of Work interface. Any aggregate root entities could implement my repository interfaces, and be a top level child off my Unit Of Work interface.

    The only problem with this I can think of is that I can only have one unit of work interface, because I have only one context per service. That's the single biggest drawback, although I'm not really sure it's going to be a problem.

    Client Side

    Once you've created your service, it publishes metadata that describes the entities it exposes to clients. As I've stated, I want to use MVVM pattern to implement clients and client views on the entities.

    You can use DataSvcUtil.exe to create entity types which implement INotifyPropertyChanged. You should then be able to implement viewmodels on top of these entities that subsequently receive notification of changes to the underlying entities and inform the UI of changes as and when necessary.

    The exact approach to implementing ViewModels that speak to entities I'm not clear on yet.

    One Big Issue I Have Overlooked

    The biggest drawback to relying on a WCF data service for implementing MVVM is that it does not support a 'push' model of change notification. That is, if data is changed on your service side model, you will not see this change until you next query (i.e. pull) the same data again on the client.

    I assume this means then that, if a client queries the service for entities and that data is edited, the last data sent back to the service always wins regardless of how long ago it was pulled.

    It also means that I can't dynamically update say, a client user interface as the entity model the service exposes changes unless I, say, poll the service every interval and get any entities whose version numbers have changed.

    Another alternative might be to also expose the client-side model through an OData service. When an update happens and the service receives a change update, the service automatically queries the change set and writes the same changes to all connected client OData services. That's thinking out loud and I'm not really sure about this approach.

    Does anyone have any thoughts on change notification with OData Services - whether it is necessary and if there are any implementations out there already that incorporate 'push' change notification in some way?




    • Edited by Markos101 Saturday, November 23, 2013 11:02 AM
    Saturday, November 23, 2013 10:58 AM
  • Hi,

    >>Does anyone have any thoughts on change notification with OData Services - whether it is necessary and if there are any implementations out there already that incorporate 'push' change notification in some way?

    Unfortunately, there is no functionality in Data Services that would do this kind of notification for you - if this is a hard requirement for you, you would need to implement something to do this for you. If you think about what Data Services is (an HTTP REST based service) this type of notification from the server to the client would violate the spirit of a REST based service. It would require the service to maintain significant amounts of cross-request state that HTTP servers typically don't do.

    There is something called WCF callbacks. Take a look at the example here.

    http://www.codeproject.com/Tips/348389/Basic-WCF-Call-plus-Callback-example

    Perhaps this is something that you can use to solve this issue.

    Regards.


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Monday, November 25, 2013 9:42 AM
  • Hi Fred,

    Thanks for your response.

    I understand the concept of WCF duplex services (and actually I'm not sure you can implement a REST web service that is duplex, as you say it seems to violate the principle of REST). And as soon as we move into the realm of duplex services, we seem to lose the REST interoperability.

    WCF RIA Services

    WCF RIA Services (http://msdn.microsoft.com/en-us/library/ee707344(v=vs.91).aspx) seem to be close to the solution I'm looking for, in that they create 'domain clients' and 'domain services'. The domain service accesses the persisted data through some DAL, and you can define POCO classes that abstract you from the physical storage medium (see http://msdn.microsoft.com/en-us/library/gg602754(v=vs.91).aspx).

    This seems to be based upon Silverlight being used as the presentation layer on clients, with the 'application logic' being the same on both the service and the client.

    I'm currently looking at whether there is something that prevents the client-side generated proxies from being used with WPF, rather than Silverlight, and how if I make a data change on one client when or how this is pushed to other clients.

    It might be that some WCF data services equivalent could be produced.

    Any thoughts, Fred?


    • Edited by Markos101 Saturday, November 30, 2013 7:09 PM
    Saturday, November 30, 2013 7:08 PM
  • WCF Data Services support Etags on entities that could be used to guard against the last win update scenario you described. Currently Microsoft is working on version 6.0 of WCF Data Services implementing OData V4. An interesting part of the V4 protocol is the ability to request only changes since last request. Although this is still a pull model it could satisfy your needs, maybe together with etags. Regards Uffe
    • Edited by Uffe Lauesen Sunday, December 01, 2013 3:26 PM Line breaks lost
    Sunday, December 01, 2013 3:25 PM
  • Hi Fred/Uffe,

    Just an update - I have been experimenting with code-first Entity Framework today just as a starting point to create a model to expose via the OData service described. Code first is excellent by the way, very easy to use.

    I've had some further thoughts about change notification, which as I say would be required to immediately inform all connected clients of changes made by a one client to the server entity model. A 'pull' or 'push' implementation is possible:

    • Uffe's suggestion of pulling only changes since last update is one possibility - even without OData V4 this could be implemented by always storing a revision number against each property value (or e.g. a timestamp). You just increment this revision number each time a change is made. You can merge in the changes by making changes on the client if the server has later revision numbers.
    • As an alternative to the polling pull model of getting changes above, a push model I think could be updated by another OData Service hosted on the client, to which the server connects. This is better than duplex because it keeps the REST model intact. Under this arrangement the server would expose a central WCF data service from which clients would pull data. The server would connect to each client WCF data service (which would expose the client's in-memory entity model of the entity model on the server). As an update message was received from a client, the server would send those updates to each client hosted WCF service with the same changes.

    With multiple connected clients, I will have to override the IUpdatable implementation on the server WCF data service in some way to keep a record of entity changes (create, update, delete) and then fire them off to each client (other than the one which sent the changes) to 'push' the changes immediately to each client.

    Anyway those are my initial thoughts - although to be honest I'm wondering now if immediately informing connected clients of changes is overkill anyway and whether we can just allow a client to get updated of changes by other clients only when they query the server next when it needs that updated data in normal functioning.

    I'll write a blog entry on what I end up doing and I will post a link back here in any case.


    • Edited by Markos101 Sunday, December 01, 2013 8:38 PM
    Sunday, December 01, 2013 8:33 PM