locked
Validate or Passthrough - Best Practice? RRS feed

  • Question

  •  

    Hi,

     

    I have a scenario where my data supplier delivers the exact same data structure/format as the trading partner expect. These interchanges are medium complex flat files, tempting me skip schema creation in biztalk and just passthrough the interchanges. At this point there's no demands on error handling or other processes - just the data delivery to deal with.

     

    What is best practice in this scenario, something tells me that creating schemas and ensuring the Bizalk validation is the right thing to do.

     

    Please give me your arguments for and against using the passthrough when possible. I might end up needing some arguments for one or the other when someone wants an explanation for my choice Smile

    Tuesday, August 5, 2008 2:20 PM

Answers

  • So BizTalk is acting only as a transportation bus that picks up files in one place, and drops them off in another?

     

    Clearly nothing in that scenario *requires* you to use anything but passthrough.  However, while the data structure is always fixed, are you completely confident in the data itself?  Given that it is fairly easy to create flat file schemas using the wizard, I'd still be tempted to build and deploy one just to have it handy in case future validation, exception handling, or processing needs arise.  This also gives you greater future flexibility if your upstream system ever decides to publish something beyond flat files and you need to deploy a mapping.

    Tuesday, August 5, 2008 3:04 PM
    Answerer
  • I usually try to look at this from a requirements perspective.  For instance we have some areas within the business that like having a schema to validate the data before it is sent to trading parters.  From a QA perspective, they would like to catch bad data before it leaves our organization.

     

    Another scenario I run into is that some other systems (a large 3 letter ERP that will remain nameless) feel that adding a schema is like duplicating business logic.  I don't quite agree here, but if that type of system already has processes in place to deal with data that does not conform to a spec then all the power to them and BizTalk is just the transport engine.

     

    Some people feel that adding a schema provides value, others think it gets in the way.  But more often then not there are political motives behind the 2nd option.

     

    Tuesday, August 5, 2008 11:18 PM
    Answerer

All replies

  • So BizTalk is acting only as a transportation bus that picks up files in one place, and drops them off in another?

     

    Clearly nothing in that scenario *requires* you to use anything but passthrough.  However, while the data structure is always fixed, are you completely confident in the data itself?  Given that it is fairly easy to create flat file schemas using the wizard, I'd still be tempted to build and deploy one just to have it handy in case future validation, exception handling, or processing needs arise.  This also gives you greater future flexibility if your upstream system ever decides to publish something beyond flat files and you need to deploy a mapping.

    Tuesday, August 5, 2008 3:04 PM
    Answerer
  • transportation bus - exactly.

    I totally agree with you on the future flexibility issue. Still, you could argue that the schema development by then would be more relevant than now.

    I'll have to be honest, Richard. You said:

    "Given that it is fairly easy to create flat file schemas using the wizard"

    In the first place i wanted to make the schema just for fun, but it has caused me som many problems, and that is why im here asking - Is validation really nessecary for transportation purposes only Smile

    Tuesday, August 5, 2008 7:07 PM
  • I usually try to look at this from a requirements perspective.  For instance we have some areas within the business that like having a schema to validate the data before it is sent to trading parters.  From a QA perspective, they would like to catch bad data before it leaves our organization.

     

    Another scenario I run into is that some other systems (a large 3 letter ERP that will remain nameless) feel that adding a schema is like duplicating business logic.  I don't quite agree here, but if that type of system already has processes in place to deal with data that does not conform to a spec then all the power to them and BizTalk is just the transport engine.

     

    Some people feel that adding a schema provides value, others think it gets in the way.  But more often then not there are political motives behind the 2nd option.

     

    Tuesday, August 5, 2008 11:18 PM
    Answerer
  • Thank you for that account.

     

    None of you guys mention performance issues. Maybe its a misconception but doesnt passthrough improve tranmission time significantly?

     

    Its not really relevant for me, as my interchange is a daily batch, but generally speaking wouldnt passthrough be preferred regarding performance?

    Wednesday, August 6, 2008 7:08 AM
  • Passthrough would definately a better performing option.  For me, this also gets worked into the requirements.  If the file is smaller and the business provides value in having a schema, then I will do it.

     

    If the file is larger and the business does not see value in a schema then I will not use one as it will kill performance.  If you try to apply a schema to a file that is in  hundreds of mb, you can count on brining down your sql server.

    Wednesday, August 6, 2008 4:08 PM
    Answerer