locked
debatching fixed length flat file RRS feed

  • Question

  • Hello,  I know there are many articles and posts about this and I've tried a few different approaches, but I cannot get it to work.  I started with a flat file schema that has a header with multiple records in it.  I used the flat file schema wizard and it worked great.  I'm able to parse the flat file into an xml file with 1 flat file creating 1 xml file with the multiple records.  Now I want to debatch the flat file into many xml files with the header and 1 record in each.  I've tried...

    1. Taking the existing schema, copying it twice (one for the header, one for the records), modify the header schema to only contain the schema, and changing the Envelope = true.  Then modifying the records schema to only contain the record.  Creating a recv pipeline with 2 flat file disassemblers.  1st disassembler with Document Schema pointing to the records schema and the second disassembler with Document schema pointing to the records schema and the header schema pointing to the header schema.
    2. Taking the original schema (only 1 schema file now), at the 'schema' properties I set 'Allow Message Breakup at Infix Root' = 'Yes', and at the Record schema set the 'Max Occurs' to '1'.
    3. Taking the existing schema, copying it twice (one for the header, one for the records), modify the header schema to only contain the schema.  Then modifying the records schema to only contain the record.  Creating a recv pipeline with 1 flat file disassembler.  Setting the 1 disassembler with Document Schema pointing to the records schema and the header schema pointing to the header schema.

    None of these has worked for me.  

    This is my original flat file schema created from the wizard that was working fine as creating one xml file with multiple records.  The AIMFileDetails is my header and the AIMFileMember is my record.


    Here is my Header Schema

    I can't upload the third picture of the record schema but it has the

    <Schema>

    AIMfile (root)

    AIMFileMember as a child of AIMfile (with all the member props like firstname, etc)


    What is the simplest way to just take my original flat file schema and change it to debatch?  Can I really just change a couple of properties?  Or should I create 2 different schemas one for the header and one for the record?  If so, do I need to use the wizard again to create each one?

    I know this should be simple but I'm clearly missing something or just getting impatient.  I've done it easily with a typedpolling schema.

    Thank you in advance for your expert advice!

    Jean


    jRenae.s

    Tuesday, October 7, 2014 9:22 PM

Answers

  • update:

    I finally got my debatching working thanks to this article...

    http://geekswithblogs.net/benny/archive/2006/02/05/68249.aspx

    The very last thing I was missing was to set the MaxOccurs property on mt record schema to 1.

    However, it is not preserving my header for me, which I do need.

    I do have the FlatFileDisassembler property 'Preserve header' = true.

    At this point I don't have an assembler as I just wanted to prove that I could indeed debatch a 5 record flat file into 5 separate xml output messages, and this sentence was stated in the article I referenced above.  So maybe that is my issue?

    To use the preserved header, you need to set the Header schema property of the flat file assembler to the same header schema you used in the disassembler.


    jRenae.s

    • Marked as answer by jRenae.s Friday, October 10, 2014 5:51 PM
    Thursday, October 9, 2014 5:13 PM

All replies

  • to continiue...here is the record schema

    I am wondering if I need the 'AIMFileMembers' (plural) as part of this schema.  Notice that it exists in the original flat file schema but not in my record schema.


    jRenae.s

    Tuesday, October 7, 2014 9:25 PM
  • Hi,

    Let me just re-iterate you requirement - you mentioned that you want to debatch the flat file into many xml files with the header and 1 record in each, and as per the above given screen shots you already have two schemas created (Header & Record Schemas).

    IMO, follow the below steps to debatch the incoming flat file:

    1. Create a receive pipeline with Flat File Disassembler component.

    2. Set the below properties of flat file disassembler component:

        Document Schema - <Your Record Schema>

        Header Schema - <Your Header Schema>

        Preserver Header - TRUE

    3. Use this pipeline at your receive location, where you are receiving the inout flat file.

    I hope this will serve your purpose.

    HTH,

    Sumit


    Sumit Verma - MCTS BizTalk 2006/2010 - Please indicate "Mark as Answer" or "Mark as Helpful" if this post has answered the question

    Wednesday, October 8, 2014 4:23 AM
  • Thank you Sumit.  Yes you are correct.  The only I hadn't done is Preserver Header - TRUE.  So I did set that and tried it again.  Still not working.  The error I'm getting is:

    'There was a failure executing the receive pipeline: "AIMtoHSP.RecAIMFFDebatched, AIMtoHSP, Version=1.0.0.0, Culture=neutral, PublicKeyToken=6f841756e8b59a59" Source: "Flat file disassembler" Receive Port: "ReceiveAIMFlatFileDebatched" URI: "C:\ports\AIMFlatFile\inDebatched\*.txt" Reason: The remaining stream has unrecognizable data.'

    I think I've defined my 2 schemas incorrectly.  Look at my original schema which was created from the wizard and works fine if I don't care about debatching.  'Details' is the header info, and 'AimFileMembers\AimfileMember' is the records. But I defined the 2 separate schemas by hand by just deleting nodes from the original, and I'm not sure if I did it correctly.   Notice that I don't have the 'AIMFileMembers' (plural) anywhere.  I'm thinking I need that in one of them.  I guess I'll will play around with that.

    Any more suggestions are greatly appreciated!

    Jean


    jRenae.s

    Wednesday, October 8, 2014 3:44 PM
  • Hi,

    You may have some trailing characters in your file. Consider removing them or handling by trailer schema: http://technet.microsoft.com/en-us/library/aa560774(BTS.70).aspx

    http://connectedthoughts.wordpress.com/2008/03/17/flat-file-disassembler-difference-between-biztalk-2004-and-2006/

    Also make sure, Child Order for Root is set properly.

    Rachit

    Please mark the post answered your question as answer, and mark other helpful posts as helpful, it'll help other users who are visiting your thread for the similar problem

    Thursday, October 9, 2014 4:27 AM
    Moderator
  • Thank you for responding, Rachit.

    I ended up recreating the header and records schema separately using the wizard.  Now I'm not getting an error, and it's processing it through the pipeline, however it's not debatching it.  I have 4 records in the flat file and still only creating one xml file in the output.

    In the pipeline properties I have the DocumentSchema set to my records schema and the HeaderSchema set to my HeaderSchema.

    Right now the 'Child Order' on the root for both header and records schemas are set to 'Postfix'.  What are they supposed to be?

    Header schema:

    Schema for records:

    So, my question is now...

    Why will this NOT debatch?  I have 2 schemas, one for header, one for records.  I have one flat file disassembler with the DocumentSchema and HeaderSchema properties set properly.  What am I missing?

    Thank you.

    Jean


    jRenae.s

    Thursday, October 9, 2014 4:28 PM
  • update:

    I finally got my debatching working thanks to this article...

    http://geekswithblogs.net/benny/archive/2006/02/05/68249.aspx

    The very last thing I was missing was to set the MaxOccurs property on mt record schema to 1.

    However, it is not preserving my header for me, which I do need.

    I do have the FlatFileDisassembler property 'Preserve header' = true.

    At this point I don't have an assembler as I just wanted to prove that I could indeed debatch a 5 record flat file into 5 separate xml output messages, and this sentence was stated in the article I referenced above.  So maybe that is my issue?

    To use the preserved header, you need to set the Header schema property of the flat file assembler to the same header schema you used in the disassembler.


    jRenae.s

    • Marked as answer by jRenae.s Friday, October 10, 2014 5:51 PM
    Thursday, October 9, 2014 5:13 PM
  • 1) 'Child Order' on the root set to 'Prefix' is fine, check the 'Child Order' set on the record it should be 'infix'.<o:p></o:p>

    2) I think you might need to set a unique Tag Identifier property for each of the schemas, and also prefix the same tag identifier on the records, so that the flat file parser can distinguish the header and record.<o:p></o:p>

    Follow:- http://bennykmathew.wordpress.com/category/integration/biztalk/pipeline-biztalk/

    Rachit

    Please mark the post answered your question as answer, and mark other helpful posts as helpful, it'll help other users who are visiting your thread for the similar problem 


    Thursday, October 9, 2014 5:15 PM
    Moderator
  • Rachit,  when you say 'prefix the same tag identifier on the records, so that the flat file parser can distinguish the header and record', it seems you mean for me to manipulate the incoming flat file, which I cannot do.  That file comes from an outside source so I have no control over it.

    Is that indeed what you mean?

    Thanks for your input!

    Jean


    jRenae.s

    Thursday, October 9, 2014 5:36 PM