none
SOA Data Aggregation Best Practices RRS feed

  • Question

  • I'm just starting to think about implementing a SOA in my organisation and have a particular query/problem I was wondering if people had any experiences or best practices with, relating to combining data from different services...

    For example: Say I have a service which retrieves customer information such as GetCustomerInfo. I call this service passing in an id, and it will return various customer properties such as Name, OpenedData, Status etc which I could then map to a  .NET class with public properties for use in my application.

    I now design a new line-of-business app to store customer referrals. This app will obviously have its own database schema, and as part of this needs to store the customer being referred.

    In the past,  I would probably present a list of customers via a drop-down-list to the user and then store the customerId as a foreign key in the local database. When it comes time to display the record, I would join to the master customer database using the customerId foreign key to get the properties I need (name etc).

    I might also need to display a list of referrals to the user, along with the customer name in which case a simple join to the master customer database would give me all the information I need.

    In an SOA world, where services boundaries are explicit I would not be able to directly join with the customer database, so what options are available to me?

    In the purest SOA sense, I guess I would need to call GetCustomerInfo(id) (or GetCustomerName(id)) for every referral record I display that also needs to display the customer name, but this could result in a 100 or more calls to the service and would have hideous performance.

    The second solution I can think of would be to also store the customer info I need in my custom database (along with the id), but this means I'd need some kind of sync-schedule setup to ensure this data was up to date?

    Does anyone have any practical experience of dealing with this kind of issue? Any advice would be greatly welcomed...

    Regards,

    Dan.

    Wednesday, September 20, 2006 2:25 PM

Answers

All replies

  • Entity aggregation services can be a solution for this, but there are many more design decisions and solutions.

    In Issue 8 of The Architecture Journal are some good articles about data integration and on MSDN is a whitepaper "SOA Challenges: Entity Aggregation".

    Wednesday, September 20, 2006 6:05 PM
  •  dan_lewis wrote:


    In the purest SOA sense, I guess I would need to call GetCustomerInfo(id) (or GetCustomerName(id)) for every referral record I display that also needs to display the customer name, but this could result in a 100 or more calls to the service and would have hideous performance.

    Doing this will make your service interaction very chatty and will not be very beneficial both in terms of performance and in SOA principles

     dan_lewis wrote:

    The second solution I can think of would be to also store the customer info I need in my custom database (along with the id), but this means I'd need some kind of sync-schedule setup to ensure this data was up to date?

    Does anyone have any practical experience of dealing with this kind of issue? Any advice would be greatly welcomed...

    Though it is best to cache immutable data it is also OK to cache data that relatively stable as long as your application knows it is not the master of the data. In regards to updating the data you can use what I call "Inversion of Communication" pattern (pardon the "patern"alism ;) - but I am writing an SOA patterns book so I am really thinking in patterns these days). This pattern essentially means that the customer service can publish (either upon change or on in a cyclic manner) changes in customer records (note that since not all services are online all the time the service should also be able to send customer data on request). Interested consumers (other services or applications) can then cache that data until they will needed. Note that you have to consider carefully if you publish more volatile data (e.g. VIP status) in contracts like these

    HTH

    Arnon

    Wednesday, September 20, 2006 9:36 PM
  • Thanks both for the suggestions/advice. It certainly gives me something to think about!

    Dan.
    Thursday, September 21, 2006 9:27 AM