locked
BRE pipeline framework for EDI and XML message validation in biztalk RRS feed

  • Question

  • Hi,

    I am working on an application in biztalk to validate an input EDI message using BRE pipeline framework. May I know hoe different BRE pipeline framework is compared to using MicrosoftRuleEngine .DLL namespace in my custom pipeline component.

    My goal is to validate an EDI X12 837 transaction according to HIPAA guidelines.  Inorder to achieve this, where can I use BRE pipeline framework.

    Friday, April 22, 2016 9:51 AM

Answers

  • Hi Naveen

    It is much easier to use the BRE pipeline framework for BRE policy execution within your custom pipeline as it contains many out-of-the-box utility vocabularies. For example you have vocabularies for reading/promoting message context properties - this will save you a lot of development time.You can find details of available utilities here-

    https://adventuresinsidethemessagebox.wordpress.com/tag/biztalk-bre-pipeline-framework/ - "the most important features are the ability to make use of XML and SQL facts to perform lookups or updates within the message body and to SQL databases, the ability to write out rule tracing data to the file system when trying to debug your policies, the addition of BizTalk 2013 context properties to enumerations"

    Plus, it is tested by the community on different projects, so will be more fool-proof.

    With this framework, you will get the BREPipelineComponent which you can then use as a custom component within your custom pipeline, say within the Decode stage.

    Ref-

    https://connectedpawns.wordpress.com/2014/01/03/bre-pipeline-framework-example-promote-distinguished-properties/


    Thanks Arindam





    Friday, April 22, 2016 10:20 AM
    Moderator
  • Hi Naveen,

    Well! All the advantages of the framework are mentioned on the home page of the codeplex. I think no one can describe the benefits then owner of the framework itself. :) 

    "The BizTalk BRE Pipeline Framework leverages the Business Rules Engine (BRE) to abstract away logic to be exercised in BizTalk pipelines thus catering for increased development flexibility and agility, especially when dealing with rapidly changing requirements, and easier deployments.

    The design goals of the BRE Pipeline Framework are as below

    • Abstract pipeline component logic into the BRE, allowing for logic to be changed at runtime without having to redeploy BizTalk artefacts.
    • Provide a collection of reusable common functions that are exercised in pipellines so that developers can concentrate on logic rather than plumbing.  Said functions will be encapsulated by business analyst friendly BRE vocabularies making for very readable rules.
    • Provide an extensible framework that allows for developers to implement their relevant custom requirements if not catered for out of the box in the desired manner.
    • Provide a means for developers to easily understand rules execution logic by providing appropriate debug tracing information when required.

    The BRE Pipeline Framework allows you to inspect and manipulate BizTalk message content and context within a pipeline.  It allows you to make use of XML and SQL based BRE vocabularies as you would normally use them, provides a collection of .NET based BRE vocabularies which should satisfy most of your message content and context assessment/manipulation requirements, and allows you to create custom .NET classes and corresponding vocabularies to extend the framework to fulfill your custom requirements."

    On how to use refer the documentation: http://brepipelineframework.codeplex.com/documentation

    If you don't want anything fancy and it's a simple implementation, you can always write your own component by adding reference to Microsoft.RulesEngine.dll.

    Samples: 

    Validating Incoming Data Using the BizTalk Business Rules Engine

    Data validation using the BRE


    Rachit Sikroria (Microsoft Azure MVP)

    Friday, April 22, 2016 12:29 PM
    Moderator

All replies

  • Hi Naveen

    It is much easier to use the BRE pipeline framework for BRE policy execution within your custom pipeline as it contains many out-of-the-box utility vocabularies. For example you have vocabularies for reading/promoting message context properties - this will save you a lot of development time.You can find details of available utilities here-

    https://adventuresinsidethemessagebox.wordpress.com/tag/biztalk-bre-pipeline-framework/ - "the most important features are the ability to make use of XML and SQL facts to perform lookups or updates within the message body and to SQL databases, the ability to write out rule tracing data to the file system when trying to debug your policies, the addition of BizTalk 2013 context properties to enumerations"

    Plus, it is tested by the community on different projects, so will be more fool-proof.

    With this framework, you will get the BREPipelineComponent which you can then use as a custom component within your custom pipeline, say within the Decode stage.

    Ref-

    https://connectedpawns.wordpress.com/2014/01/03/bre-pipeline-framework-example-promote-distinguished-properties/


    Thanks Arindam





    Friday, April 22, 2016 10:20 AM
    Moderator
  • Hi Naveen,

    Well! All the advantages of the framework are mentioned on the home page of the codeplex. I think no one can describe the benefits then owner of the framework itself. :) 

    "The BizTalk BRE Pipeline Framework leverages the Business Rules Engine (BRE) to abstract away logic to be exercised in BizTalk pipelines thus catering for increased development flexibility and agility, especially when dealing with rapidly changing requirements, and easier deployments.

    The design goals of the BRE Pipeline Framework are as below

    • Abstract pipeline component logic into the BRE, allowing for logic to be changed at runtime without having to redeploy BizTalk artefacts.
    • Provide a collection of reusable common functions that are exercised in pipellines so that developers can concentrate on logic rather than plumbing.  Said functions will be encapsulated by business analyst friendly BRE vocabularies making for very readable rules.
    • Provide an extensible framework that allows for developers to implement their relevant custom requirements if not catered for out of the box in the desired manner.
    • Provide a means for developers to easily understand rules execution logic by providing appropriate debug tracing information when required.

    The BRE Pipeline Framework allows you to inspect and manipulate BizTalk message content and context within a pipeline.  It allows you to make use of XML and SQL based BRE vocabularies as you would normally use them, provides a collection of .NET based BRE vocabularies which should satisfy most of your message content and context assessment/manipulation requirements, and allows you to create custom .NET classes and corresponding vocabularies to extend the framework to fulfill your custom requirements."

    On how to use refer the documentation: http://brepipelineframework.codeplex.com/documentation

    If you don't want anything fancy and it's a simple implementation, you can always write your own component by adding reference to Microsoft.RulesEngine.dll.

    Samples: 

    Validating Incoming Data Using the BizTalk Business Rules Engine

    Data validation using the BRE


    Rachit Sikroria (Microsoft Azure MVP)

    Friday, April 22, 2016 12:29 PM
    Moderator
  • Thank you :)

    Tuesday, April 26, 2016 12:39 PM
  • Thank you Arindam.
    Tuesday, April 26, 2016 12:39 PM
  • My goal is to validate an EDI X12 837 transaction according to HIPAA guidelines.  Inorder to achieve this, where can I use BRE pipeline framework.

    BRE yes.  But what exactly are you trying to do?  What you say above doesn't really mean anything.

    An 837 is automatically validated against L1 and L2 (which technically aren't HIPAA standards).  L3-L5 are often business specific situations which are handled by an existing app.

    So, before you spend too much time deciding between PFx, custom implementation or an Orchestration, you need to find out from the business exactly, precisely and specifically what they want validated.

    Tuesday, April 26, 2016 9:27 PM
    Moderator
  • Hi,

    Yeah you are right. L1 and L2 are automatically validated.

    from l3 - l5 , we are using EDIFECS management tool. that must be the existing App you should be talking about.

    I would like to eliminate EDIFECS mgmt tool replacing it with the app I create using BRE pipeline framework.

    CIAO.

    Wednesday, April 27, 2016 4:53 AM
  • Hi Arinadam,

    Thanks a lot for your inputs.

    Could you help me create an application to validate an XML or EDI message based on the data in the fields?

    I don't want any transformation of message. Just route the message to destination if the data in XML or EDI file matches the requirement.

    To achieve this, how can we create the rules using BRE pipeline framework vocabularies and call those rules inside my biztalk application. 

    How to use the BREpipeline framework component.

    Regards

    Naveen

    Tuesday, May 24, 2016 6:53 AM
  • Hi Naveen

    To validate your XML, you will have to first get full list of validation rules from business - for example, what are mandatory fields, are there some validation rules based on some values in the message perhaps, etc.

    Once you have those in place, you will have to create a BRE policy that contains each of your business rules as a separate BRE rule.

    To create each of these validation rules, you can leverage some of the BRE Pipeline Framework vocabularies(more on this below), or write your own custom .NET helper methods and publish those as BRE vocabularies. For checking if a field exists or not, it is very simple to setup a XPATH based vocabulary that fetches the field you need to check for values. But, for more complex validations, you will proably have to write your own .NET helper class that reads your input XML field/document, and performs the validation. The methods on your custom class can then be published as vocabularies and then used from your BRE validation rules. You may want to have a separate header section in the message you pass to BRE where you can write the rule execution results. Maybe, in your ReceivePort map the input message to an envelope schema that contains an additional ValidationHeader XML record in addition to your input message. Your BRE rule Actions can then write to the ValidationHeader section the execution results of each of your BRE rule.

    One such validation example can be found here-

    https://seroter.wordpress.com/2009/11/11/validating-incoming-data-using-the-biztalk-business-rules-engine/

    Basically, the BREPipelineFramework can make your job easier by providing some out-of-the-box vocabularies for getting XPATH values, etc., but you will still need to write your custom rules and policy. Calling this policy from your pipeline will certainly be simple due to the BREPipelineFramework pipeline component. But designing the policy is something that you will have to do yourself.

    While creating your validation rules in your policy, you can take help of some of the out of the box vocabularies shipped with BREPipelineFramework. Summary explanation of out of the box vocabularies -

    After completing the installation instructions you should three BRE Pipeline Framework related vocabularies (you might potentially have multiple versions of each one) published to the rules engine database that you can view in the Business Rules Composer as below.

    BREPipelineFramework - This vocabulary for the most part can only be used in InstructionLoaderPolicy BRE policies.  The one exception to this rule is the ApplicationContext vocabulary definition which can be used in ExecutionPolicy BRE policies as well, and can be used in rules to evaluate the value of the ApplicationContext on the pipleine properties.  Other vocabulary definitions that can only be used in InstructionLoaderPolicy BRE policies can be used to inspect the BizTalk message (through regex matching, string finds, XPaths) for condition evaluation, and to assert .NET, XML, or SQL based facts, or to override the ExecutionPolicy to be called, or to override the ApplicationContext to be passed to the ExecutionPolicy.
     BREPipelineFramework.SampleInstructions.ContextInstructions - This vocabulary can only be used in ExecutionPolicy BRE policies, and contains vocabulary definitions to get or set all BizTalk out of the box context properties through enumeration, to get or set custom context properties, and to remove context properties.
     BREPipelineInstructions.SampleInstructions.HelperInstructions - This vocabulary provides helper type functionality to evaluate and manipulate message content and context, as well as common helper functionality such as string concatenation and formatting, date time rounding, guid generation, get the current date time, or throw an exception with a custom message. 

    https://brepipelineframework.codeplex.com/documentation


    Thanks Arindam






    Wednesday, May 25, 2016 7:17 AM
    Moderator
  • Thank you arindam. This helps a lot  :)

    Regards Naveen Alokam

    Wednesday, June 8, 2016 10:08 AM