none
Canonical Schema - why should I RRS feed

  • Question

  • Hi there, 

    I can't see any advantage in using Canonical Data model and Canonical Schema in BizTalk. 

    In my opinion it justs adds to complexity of the project, normally I would have just two schemas and in the canonical model I need to maintain three schemas. 

    Also lets consider the following example:

    We receive a schema that contains the following:

    PONumber

    Customer FirstName 

    Customer Last name

    Product 

    Quantity 

    TotalPrice

    The Outgoing Schema requires:

    PONumber

    Customer FullName (Concatenation used)

    Product 

    Quantity 

    TotalPrice

    I would simply develope a map between these two using a concatenate functoid and that's it. 

    Why to complicate this by introducing another element in for of additional schema that you need to maintain.

    ???


    God bless you all :)

    Saturday, September 19, 2015 5:47 PM

Answers

  • Hi btsAdministrator,

    A canonical schema is a design pattern, which is applied within a service-oriented paradigm, and within the BizTalk Server context to establish loose coupling between systems. Through performing transformation of messages from one system to a canonical schema and from a canonical schema to a message of another system, systems have no direct relationship with each other. A canonical schema can also be viewed as an internal schema in BizTalk and aid you in structuring your solution through the best practice of creating separate projects for maps, orchestrations, internal and external schemas.

    The term canonical schema is often used to describe the creation of schemas internal to an integration mechanism such as BizTalk.

    Use of canonical schemas is widely regarded as a best practice, as it decouples your BizTalk flow control mapping from any 'other' system's schemas (other system here could be internal to your organisation or external to it, e.g. a supplier, customer or partner system). This way, if any of the systems integrated via BizTalk change, it is just the external schemas, and maps to the canonical schemas which need to be changed. It also prevents foreign conventions, naming and hierarchy differences inherent in external schemas from leaking into your internal BizTalk artefacts.

    Generally, transformation of incoming messages to a canonical schema is done as early as possible e.g. on a receive, and similarly, transformation out of canonical done as late as possible, e.g. on a send port map.

    A common scenario for Canonical Schemas (CS) is where a single orchestration or message flow is common to multiple trading parties (e.g. you may have many suppliers with different systems, however, all of them submit invoices for processing). In this case, each new supplier system just needs to be integrated with your CS - no new processing logic needs to be added or duplicated - CS can actually reduce the overall effort in such instances. Another example of where CS are vital is where your business IS switching of messages - e.g. a Medical industry switch will have many doctor and practice systems sending authorisation requests and invoices and these need to be mapped and routed to multiple medical fund (medical aid) systems.

    And For What It's Worth:

    1. In my opinion, CS make most sense in an when BizTalk is the end-end solution in an EAI or ESB scenario, e.g. direct integration of 2 or more line of business systems. Otherwise, if BizTalk is just one endpoint on a larger corporate ESB, then it probably makes sense to use the corporate ESB schemas internally, and hence map external schemas directly to the ESB schemas (i.e. no need for another set of CS within BizTalk, provided that you have a good change management / version control mechanism across your enterprise).
    2. If standard schemas (e.g. EDIFACT) exist for your industry, it is moot as to whether it is a goal to adopt these as internal CS. In general these may conflict with the meaning of Canonical as being 'simple', as industry schemas often need to be verbose in order to model all flavours and 'edge cases' of the document). Personally I would ensure that I have a mapping to / from said industry schemas, but would use a custom schema internally.

    Reference: http://stackoverflow.com/questions/21477147/biztalk-internal-and-external-schemas

    I would suggest you please have a look into below useful articles that will explain benefits of the canonical pattern and why solutions should implement it.

    A brief note on Canonical Schemas

    Thoughts on the Canonical Messaging Pattern

    Create a Canonical Schema – Step by Step

    Should i use Canonical schema for my requirement


    Thanks, If my reply is helpful please mark as answer or vote as helpful.

    Saturday, September 19, 2015 9:10 PM
    Moderator

All replies

  • Hi btsAdministrator,

    A canonical schema is a design pattern, which is applied within a service-oriented paradigm, and within the BizTalk Server context to establish loose coupling between systems. Through performing transformation of messages from one system to a canonical schema and from a canonical schema to a message of another system, systems have no direct relationship with each other. A canonical schema can also be viewed as an internal schema in BizTalk and aid you in structuring your solution through the best practice of creating separate projects for maps, orchestrations, internal and external schemas.

    The term canonical schema is often used to describe the creation of schemas internal to an integration mechanism such as BizTalk.

    Use of canonical schemas is widely regarded as a best practice, as it decouples your BizTalk flow control mapping from any 'other' system's schemas (other system here could be internal to your organisation or external to it, e.g. a supplier, customer or partner system). This way, if any of the systems integrated via BizTalk change, it is just the external schemas, and maps to the canonical schemas which need to be changed. It also prevents foreign conventions, naming and hierarchy differences inherent in external schemas from leaking into your internal BizTalk artefacts.

    Generally, transformation of incoming messages to a canonical schema is done as early as possible e.g. on a receive, and similarly, transformation out of canonical done as late as possible, e.g. on a send port map.

    A common scenario for Canonical Schemas (CS) is where a single orchestration or message flow is common to multiple trading parties (e.g. you may have many suppliers with different systems, however, all of them submit invoices for processing). In this case, each new supplier system just needs to be integrated with your CS - no new processing logic needs to be added or duplicated - CS can actually reduce the overall effort in such instances. Another example of where CS are vital is where your business IS switching of messages - e.g. a Medical industry switch will have many doctor and practice systems sending authorisation requests and invoices and these need to be mapped and routed to multiple medical fund (medical aid) systems.

    And For What It's Worth:

    1. In my opinion, CS make most sense in an when BizTalk is the end-end solution in an EAI or ESB scenario, e.g. direct integration of 2 or more line of business systems. Otherwise, if BizTalk is just one endpoint on a larger corporate ESB, then it probably makes sense to use the corporate ESB schemas internally, and hence map external schemas directly to the ESB schemas (i.e. no need for another set of CS within BizTalk, provided that you have a good change management / version control mechanism across your enterprise).
    2. If standard schemas (e.g. EDIFACT) exist for your industry, it is moot as to whether it is a goal to adopt these as internal CS. In general these may conflict with the meaning of Canonical as being 'simple', as industry schemas often need to be verbose in order to model all flavours and 'edge cases' of the document). Personally I would ensure that I have a mapping to / from said industry schemas, but would use a custom schema internally.

    Reference: http://stackoverflow.com/questions/21477147/biztalk-internal-and-external-schemas

    I would suggest you please have a look into below useful articles that will explain benefits of the canonical pattern and why solutions should implement it.

    A brief note on Canonical Schemas

    Thoughts on the Canonical Messaging Pattern

    Create a Canonical Schema – Step by Step

    Should i use Canonical schema for my requirement


    Thanks, If my reply is helpful please mark as answer or vote as helpful.

    Saturday, September 19, 2015 9:10 PM
    Moderator
  • As has already been explained to you, Canonical Schema is a pattern and not something you would deploy everywhere. It has a specific use when dealing with multiple partners where every partner would use a message format different from what you would want. Secondly if you are focused on "Processes" as opposed to "Integration" then having canonical schemas will help you decouple your process from the application specifics allowing you to change or modify core applications without having to change the process.

    Canonical Schemas used in combination with BizTalk Parties, Orchestration Role Links is a very very powerful method for dealing with partners in a B2B scenario. 

    Regards.

     
    Sunday, September 20, 2015 2:10 PM
  • There are scenarios where a canonical pattern can be useful.

    • There is a reasonable chance there will be multiple producers and consumers of a business message. >2 to >2
    • Creating a business process that is not tightly bound to an external or proprietary format.
    • Make unwieldy, non-normalized, or just weird formats easier to work with.

    If you're just integrating two slow moving but well supported apps, SAP to Oracle for instance, a canonical pattern will likely introduce complexity for little benefit.

    Sunday, September 20, 2015 2:46 PM
    Moderator
  • Hi,

    Canonical schema is a design pattern and you need to make a decision whether it is useful for you or not in your context. Normally when you receive different messages formats/types into BizTalk server insteade of transforming from each and every message to your LOB systemm you maintain a canonical schema and transform all messages to canonical format. And from canonical format you transform the messages specific to your LOB system so that you can maintain all changes at one place. If there is a schema change/map change on destination system all you need to do is at a single place canonical schema.

    Cheers


    JB

    Sunday, September 20, 2015 9:36 PM
  • Hi,

    It is for good design approach to have a canonical schema defined . You can have multiple operation data structure defined with canonical schema.

    Use of canonical schemas is widely regarded as a best practice, as it decouples your BizTalk flow control mapping from any 'other' system's schemas (other system here could be internal to your organisation or external to it, e.g. a supplier, customer or partner system). This way, if any of the systems integrated via BizTalk change, it is just the external schemas, and maps to the canonical schemas which need to be changed. It also prevents foreign conventions, naming and hierarchy differences inherent in external schemas from leaking into your internal BizTalk artefacts.

    Refer http://stackoverflow.com/questions/21477147/biztalk-internal-and-external-schemas

    Thanks

    Abhishek

    Monday, September 21, 2015 4:50 AM
  • Hi BTSAdministrator,

    Let us suppose we take an example of Builder who Sells 1,2 and 3 BHK to customers and his existing system is like He receives 1 BHK requirement from customer from location A(with 1 BHK msgType)

    and 2 BHK requirement from customer from location B(with 2BHK msgType)

    and 3 BHK requirement from customer from location C(with 3BHK msgType)

    so total we have 3 Recieve locations.

    Now let us assume he create a canonical schema(combination of 1,2 and 3 bhk message type)then from any one location he can receive all 3 type of messages because now he is using canonical schema which can accept any BHK type message .

    This is very small example to identify the uses of Canonical Schema.

    Other Advantages are like 

    • Less transformation maps: When you have 2 transaction types, 5 external data formats and 3 internal data formats you would need 5*3*2 = 30 maps in a traditional peer-to-peer situation. With a Canonical Data Model, you’d only need 5*2 + 3*2 = 16 maps. You can imagine that the number of maps grows exponentially with peer-to-peer maps, when introducing new transaction types, more external formats and/or more internal formats…;
    • Less subscription rules: Instead of having to subscribe all your internal systems and transaction types to all the different externally received message types, you only need to subscribe them to the Canonical Data Formats. This means less management and easier troubleshooting;
    • Less impact of schema changes: Whenever an internal or external schema needs a change, it only impacts the one map between the external message and the canonical data format or the one map between the canonical data format and the internal message. This applies to most schema changes (of course there are exceptions);
    • Everyone does not need to know everything: Maybe this is the most important reason. When designing a map for receiving let’s say an X12 4010 850 Purchase Order, you only need to know that format and the PO Canonical Data Format really well in order to be able to do that. You don’t need to know anything about the internal data formats (SAP, Oracle, Dynamics, Homegrown LOB systems, etc.). The same applies to maps to and from the internal systems. The people designing these don’t need to know anything about the external data formats.

    For more details please refer 

    http://www.indafield.com/BizTalkNotes/2008/02/04/canonical-data-model-a-must-have/

    Thanks, If my reply is helpful please mark as answer or vote as helpful.

    Thanks

    Manoj


    Monday, September 21, 2015 8:49 AM