none
Flat File Debatching RRS feed

  • Question

  • Hi

    I have to debatch a flat File with the following structure

    UN1
    .
    .
    .
    UN1
    .
    .
    .
    etc.

    I would like to do the devbatching with a pipeline.

    My Schema has a root node and a record with an element inside

    root
        record
                element (where the data is inside)

    I cannot use a fully strutured schema because the the incoming flatfile have a different structure. The only characterisitca they have in common is the header record UN1.

    I tired a lot but I did not succeed. How do I have to configure the child order and the child order delimiter type for this flat file?

    Thanks for your help 

    Klaus Weber
    Wednesday, November 4, 2009 10:28 PM

Answers

  • Prefix means the delimiter preceeds the data, i.e. <delimeter><data><delimeter><data>. This is your case, I don't know why it didn't work - I'll look into it if I have spare time.

    When you changed child order to Infix, you "told" parser to extract data from both sides of delimiter, i.e.
     <data><delimeter><data><delimeter><data>

    As you can see, when the child order is Infix you get 3 <data> elements (messages in your case) off a file with 2 delimiter. The first message will have 0 bytes because the input file starts with the delimiter.
    Kiryl Kavalenka My Blog
    Wednesday, November 11, 2009 12:10 AM

All replies

  • Try this. Map it to a database all the records you have. Then create another map that will select all the records into another map. This will enable you to break the file appart into single, two, or however many messages you want because you will select the records from a stored procedure. It will do what you need. Good luck.
    Please Indicate "Mark as Answer" if a Post has Answered the Question
    Thursday, November 5, 2009 1:03 PM
  • Hi

    I tried some other things and I nearly have the solution:

    I modified my schema to

    <Schema>
           Root
              Element

    The configuration for the Child order and delimiters is:

    <schema> => Child Delimiter  = CR/LF UN1, Default Child Order = Conditional Default, Alllow Message Breakup at Infix Root = Yes
        Root     => Child Delimiter = Default Child Delimiter, Child Order= Infix


    The result is that the pipeline debtaches the interchange into the correct number of messages but deletes the UN1 in every message EXCEPT the first one.

    Thanks for your thoughts

    Klaus Weber
    Friday, November 6, 2009 2:50 PM
  • Try this:

    <Schema>
           Root (Child Order= Postfix, Delimiter = 0xD 0xA, Delimiter Type = Hexadecimal)
              Element (Tag Identifier= UN1)
                  Field


    <?xml version="1.0" encoding="utf-16"?>
    <xs:schema xmlns:b="http://schemas.microsoft.com/BizTalk/2003" xmlns="http://Test.Schemas.test.FlatFileSchema1" targetNamespace="http://Test.Schemas.test.FlatFileSchema1" xmlns:xs="http://www.w3.org/2001/XMLSchema">
      <xs:annotation>
        <xs:appinfo>
          <b:schemaInfo standard="Flat File" root_reference="Root" default_pad_char=" " pad_char_type="char" count_positions_by_byte="false" parser_optimization="speed" lookahead_depth="3" suppress_empty_nodes="false" generate_empty_nodes="true" allow_early_termination="false" early_terminate_optional_fields="false" allow_message_breakup_of_infix_root="false" compile_parse_tables="false" default_child_order="postfix" />
          <schemaEditorExtension:schemaInfo namespaceAlias="b" extensionClass="Microsoft.BizTalk.FlatFileExtension.FlatFileExtension" standardName="Flat File" xmlns:schemaEditorExtension="http://schemas.microsoft.com/BizTalk/2003/SchemaEditorExtensions" />
        </xs:appinfo>
      </xs:annotation>
      <xs:element name="Root">
        <xs:annotation>
          <xs:appinfo>
            <b:recordInfo structure="delimited" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" sequence_number="1" child_delimiter_type="hex" child_order="postfix" child_delimiter="0x0D 0x0A" />
          </xs:appinfo>
        </xs:annotation>
        <xs:complexType>
          <xs:sequence>
            <xs:annotation>
              <xs:appinfo>
                <b:groupInfo sequence_number="0" />
              </xs:appinfo>
            </xs:annotation>
            <xs:element name="Record">
              <xs:annotation>
                <xs:appinfo>
                  <b:recordInfo sequence_number="1" structure="delimited" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" tag_name="UN1" />
                </xs:appinfo>
              </xs:annotation>
              <xs:complexType>
                <xs:sequence>
                  <xs:annotation>
                    <xs:appinfo>
                      <b:groupInfo sequence_number="0" />
                    </xs:appinfo>
                  </xs:annotation>
                  <xs:element name="Field" type="xs:string">
                    <xs:annotation>
                      <xs:appinfo>
                        <b:fieldInfo sequence_number="1" justification="left" />
                      </xs:appinfo>
                    </xs:annotation>
                  </xs:element>
                </xs:sequence>
              </xs:complexType>
            </xs:element>
          </xs:sequence>
        </xs:complexType>
      </xs:element>
    </xs:schema>

    Saturday, November 7, 2009 10:28 AM
  • Hi Kiryl

    Thanks for your input. With your schema I get the error " The remaining stream has unrecognizable data"

    Here I have some sample data. Perhaps my problem is easier to understand with this sample

    Source:


    UN1UNOA212345678990123                                        2222222                                              0000000
    UNH1             DESADVD  93AUNEAN004
    BGM351                                         500326                             9
    DTM13720090703102
    DTM17 20090703102
    REFON 44444444
    REFIV 333333
    NADBY 12345678990123
    NADSU 12345678990123
    NADDP 12345678990123
    CPS1
    LIN0000014004708355246EN
    PIA  1111111                             SA
    IMD                   Productinformation short
    QTY12 000000000005000PCE
    CNT2  000000000000000001
    UNT00001500000000000000
    UNZ00000516380000000000
    UN1UNOA21234567890123                                        2222222                                              0000000
    UNH1             DESADVD  93AUNEAN004
    BGM351                                         500331                             9
    DTM13720090703102
    DTM17 20090703102
    REFON 44444444
    REFIV 333333
    NADBY 12345678990123
    NADSU 12345678990123
    NADDP 12345678990123
    CPS1
    LIN0000011111111111111EN
    PIA  2222222                             SA
    IMD                   Productinformation short
    QTY12 000000000001000PCE
    CNT2  000000000000000001
    UNT00001500000000000000


    I need two files with the UN1 record at the beginning of each file


    Thanks for your help

    Klaus Weber
    Monday, November 9, 2009 1:36 PM
  • Sorry, it is still not clear whether you want "UN1" to be preserved in the output messages or not.
    If not, here is the schema:
    <Schema>
           Root (Child Order= Prefix, Delimiter = UN1, Delimiter Type = Character)
              Element
                  Field
    <?xml version="1.0" encoding="utf-16"?>
    <xs:schema xmlns:b="http://schemas.microsoft.com/BizTalk/2003" xmlns="http://BizTalk_Server_Project1.FlatFileSchema1" targetNamespace="http://BizTalk_Server_Project1.FlatFileSchema1" xmlns:xs="http://www.w3.org/2001/XMLSchema">
      <xs:annotation>
        <xs:appinfo>
          <b:schemaInfo standard="Flat File" root_reference="Root" default_pad_char=" " pad_char_type="char" count_positions_by_byte="false" parser_optimization="speed" lookahead_depth="3" suppress_empty_nodes="false" generate_empty_nodes="true" allow_early_termination="false" early_terminate_optional_fields="false" allow_message_breakup_of_infix_root="false" compile_parse_tables="false" />
          <schemaEditorExtension:schemaInfo namespaceAlias="b" extensionClass="Microsoft.BizTalk.FlatFileExtension.FlatFileExtension" standardName="Flat File" xmlns:schemaEditorExtension="http://schemas.microsoft.com/BizTalk/2003/SchemaEditorExtensions" />
        </xs:appinfo>
      </xs:annotation>
      <xs:element name="Root">
        <xs:annotation>
          <xs:appinfo>
            <b:recordInfo structure="delimited" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" sequence_number="1" child_delimiter_type="char" child_delimiter="UN1" child_order="prefix" />
          </xs:appinfo>
        </xs:annotation>
        <xs:complexType>
          <xs:sequence>
            <xs:annotation>
              <xs:appinfo>
                <b:groupInfo sequence_number="0" />
              </xs:appinfo>
            </xs:annotation>
            <xs:element name="Record">
              <xs:annotation>
                <xs:appinfo>
                  <b:recordInfo sequence_number="1" structure="delimited" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />
                </xs:appinfo>
              </xs:annotation>
              <xs:complexType>
                <xs:sequence>
                  <xs:annotation>
                    <xs:appinfo>
                      <b:groupInfo sequence_number="0" />
                    </xs:appinfo>
                  </xs:annotation>
                  <xs:element name="Field" type="xs:string">
                    <xs:annotation>
                      <xs:appinfo>
                        <b:fieldInfo sequence_number="1" justification="left" />
                      </xs:appinfo>
                    </xs:annotation>
                  </xs:element>
                </xs:sequence>
              </xs:complexType>
            </xs:element>
          </xs:sequence>
        </xs:complexType>
      </xs:element>
    </xs:schema>

    Kiryl Kavalenka My Blog
    Monday, November 9, 2009 6:27 PM
  • Hi kyril

    I would like to to preserve the UN1. I tried your schema but it does not split the interchange inside the pipeline using the flatfile dissasembler.
    Klaus Weber
    Tuesday, November 10, 2009 9:09 AM
  • Hi,

    BizTalk pipeline does not preserve the Header and Trailer information by default. If you need to use only one or two fields you could possible promote them and then you could use them later if required.

    Regards
    Vishnu
    Vishnu
    Tuesday, November 10, 2009 10:54 AM
  • Hi

    After changing the the root child order from Prefix to infix the source file has been split in 3 files. Two of them have an acceptable content (only the UN1 Tag is missing in the header row) and the third one is a 0 byte file. Where does this 0 byte file come from?


    Klaus Weber
    Tuesday, November 10, 2009 2:41 PM
  • Prefix means the delimiter preceeds the data, i.e. <delimeter><data><delimeter><data>. This is your case, I don't know why it didn't work - I'll look into it if I have spare time.

    When you changed child order to Infix, you "told" parser to extract data from both sides of delimiter, i.e.
     <data><delimeter><data><delimeter><data>

    As you can see, when the child order is Infix you get 3 <data> elements (messages in your case) off a file with 2 delimiter. The first message will have 0 bytes because the input file starts with the delimiter.
    Kiryl Kavalenka My Blog
    Wednesday, November 11, 2009 12:10 AM
  • Are you sure that MaxOccurs of "Record" element is set to 1(the default value)?

    Kiryl Kavalenka My Blog
    Wednesday, November 11, 2009 12:12 AM
  • Hi Kyril


    Yes I definitely set the MaxOccurence to 1. When I test it in Visual Studio I think the debtaching is ok because only the first message is translated to xml (=> validate instance). If I do it via pipeline the result is one file with both messages
    Klaus Weber
    Wednesday, November 11, 2009 11:47 AM