locked
Regarding debatching in flat file RRS feed

  • Question

  • My input file positional delimited structure is like first line is header from where i am generaing five record in header and from second line detail node will have five record
    0000001782008011104244600001029                                                                                <-----Header
    NL 0000000003000000000000000001200801110248020000000000000012122   10.7      <-----Record
    NL 0000000003000000000000000003200801110248020000000000000037106   11.7
    NL 0000000003000000000000000006200801110248020000000000000016934   10.7
    i have to crete debatch such like each record will have header record

    when i am testing my schema by validating instance with this file in visual studio i am able to debatch 1 header with 1 detail node;but while i am deploying that application and testing then there is no any output;its giving error that unexpected error found while looking for '\n\r\'

    Please let me know if need any clarification regarding my query
    Please help
    Regards

    Thursday, July 9, 2009 5:23 AM

Answers

  • For the header you can select Preserve Header and it puts the header XML into a message context for each debatched message. Here is an example on how to use it:
    http://seroter.wordpress.com/2007/08/23/debatching-flat-files-into-xml-with-header-intact/


    Regards,
    Thiago Almeida - http://connectedthoughts.wordpress.com
    • Proposed as answer by eliasen Sunday, July 12, 2009 8:50 PM
    • Marked as answer by Andrew_ZhuModerator Thursday, July 16, 2009 1:29 AM
    Saturday, July 11, 2009 12:46 AM
    Answerer
  • You could still build the messages in the orchestration and then submit them back to the message box. But if you don't want to use an orchestration for performance or other reasons you will have to create your own custom pipeline component.

    Your custom pipeline component would inherit from the flat file disassembler component (FFDasmComp). On the first GetNext method you would call the base.GetNext method and get the header from the message context. Then construct the final message returned by GetNext by combining the header with the body every time GetNext is called.
    There's an explanation here on the flat file disassembler class:
    http://www.codeproject.com/KB/biztalk/DebatchingFlatfile.aspx

    You will have to read on writing custom disassembler pipeline components (many blog posts and other online entries about it).
    Thiago Almeida - http://connectedthoughts.wordpress.com
    Monday, July 13, 2009 8:04 PM
    Answerer

All replies

  • can you post the schema?
    KiranMP
    Thursday, July 9, 2009 9:02 AM
  • <?xml version="1.0" encoding="utf-16"?>
    <xs:schema xmlns:b="http://schemas.microsoft.com/BizTalk/2003" xmlns="http://FRCRT.DebatchUsingSchema.Demo.FlatFileSchema3" targetNamespace="http://FRCRT.DebatchUsingSchema.Demo.FlatFileSchema3" xmlns:xs="http://www.w3.org/2001/XMLSchema">
      <xs:annotation>
        <xs:appinfo>
          <schemaEditorExtension:schemaInfo namespaceAlias="b" extensionClass="Microsoft.BizTalk.FlatFileExtension.FlatFileExtension" standardName="Flat File" xmlns:schemaEditorExtension="http://schemas.microsoft.com/BizTalk/2003/SchemaEditorExtensions" />
          <b:schemaInfo standard="Flat File" codepage="65001" default_pad_char=" " pad_char_type="char" count_positions_by_byte="false" parser_optimization="speed" lookahead_depth="3" suppress_empty_nodes="false" generate_empty_nodes="true" allow_early_termination="false" early_terminate_optional_fields="false" allow_message_breakup_of_infix_root="false" compile_parse_tables="false" root_reference="Root" />
        </xs:appinfo>
      </xs:annotation>
      <xs:element name="Root">
        <xs:annotation>
          <xs:appinfo>
            <b:recordInfo structure="delimited" child_delimiter_type="hex" child_delimiter="0xD 0xA" child_order="postfix" sequence_number="1" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />
          </xs:appinfo>
        </xs:annotation>
        <xs:complexType>
          <xs:sequence>
            <xs:annotation>
              <xs:appinfo>
                <groupInfo sequence_number="0" xmlns="http://schemas.microsoft.com/BizTalk/2003" />
              </xs:appinfo>
            </xs:annotation>
            <xs:element name="Header">
              <xs:annotation>
                <xs:appinfo>
                  <b:recordInfo structure="positional" tag_name="HH" tag_offset="0" sequence_number="1" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />
                </xs:appinfo>
              </xs:annotation>
              <xs:complexType>
                <xs:sequence>
                  <xs:annotation>
                    <xs:appinfo>
                      <groupInfo sequence_number="0" xmlns="http://schemas.microsoft.com/BizTalk/2003" />
                    </xs:appinfo>
                  </xs:annotation>
                  <xs:element name="H1" type="xs:string">
                    <xs:annotation>
                      <xs:appinfo>
                        <b:fieldInfo justification="left" pos_offset="0" pos_length="2" sequence_number="1" />
                      </xs:appinfo>
                    </xs:annotation>
                  </xs:element>
                  <xs:element name="H2" type="xs:string">
                    <xs:annotation>
                      <xs:appinfo>
                        <b:fieldInfo justification="left" pos_offset="0" pos_length="9" sequence_number="2" />
                      </xs:appinfo>
                    </xs:annotation>
                  </xs:element>
                  <xs:element name="H3" type="xs:string">
                    <xs:annotation>
                      <xs:appinfo>
                        <b:fieldInfo justification="left" pos_offset="0" pos_length="8" sequence_number="3" />
                      </xs:appinfo>
                    </xs:annotation>
                  </xs:element>
                  <xs:element name="H4" type="xs:string">
                    <xs:annotation>
                      <xs:appinfo>
                        <b:fieldInfo justification="left" pos_offset="0" pos_length="6" sequence_number="4" />
                      </xs:appinfo>
                    </xs:annotation>
                  </xs:element>
                  <xs:element name="H5" type="xs:string">
                    <xs:annotation>
                      <xs:appinfo>
                        <b:fieldInfo justification="left" pos_offset="0" pos_length="8" sequence_number="5" />
                      </xs:appinfo>
                    </xs:annotation>
                  </xs:element>
                </xs:sequence>
              </xs:complexType>
            </xs:element>
            <xs:element minOccurs="1" maxOccurs="1" name="Detail">
              <xs:annotation>
                <xs:appinfo>
                  <b:recordInfo structure="positional" tag_name="DD" tag_offset="0" sequence_number="2" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />
                </xs:appinfo>
              </xs:annotation>
              <xs:complexType>
                <xs:sequence>
                  <xs:annotation>
                    <xs:appinfo>
                      <groupInfo sequence_number="0" xmlns="http://schemas.microsoft.com/BizTalk/2003" />
                    </xs:appinfo>
                  </xs:annotation>
                  <xs:element name="D1" type="xs:string">
                    <xs:annotation>
                      <xs:appinfo>
                        <b:fieldInfo justification="left" pos_offset="0" pos_length="2" sequence_number="1" />
                      </xs:appinfo>
                    </xs:annotation>
                  </xs:element>
                  <xs:element name="D2" type="xs:string">
                    <xs:annotation>
                      <xs:appinfo>
                        <b:fieldInfo justification="left" pos_offset="0" pos_length="3" sequence_number="2" />
                      </xs:appinfo>
                    </xs:annotation>
                  </xs:element>
                  <xs:element name="D3" type="xs:string">
                    <xs:annotation>
                      <xs:appinfo>
                        <b:fieldInfo justification="left" pos_offset="0" pos_length="9" sequence_number="3" />
                      </xs:appinfo>
                    </xs:annotation>
                  </xs:element>
                  <xs:element name="D4" type="xs:string">
                    <xs:annotation>
                      <xs:appinfo>
                        <b:fieldInfo justification="left" pos_offset="0" pos_length="19" sequence_number="4" />
                      </xs:appinfo>
                    </xs:annotation>
                  </xs:element>
                  <xs:element name="D5" type="xs:string">
                    <xs:annotation>
                      <xs:appinfo>
                        <b:fieldInfo justification="left" pos_offset="0" pos_length="8" sequence_number="5" />
                      </xs:appinfo>
                    </xs:annotation>
                  </xs:element>
                  <xs:element name="D6" type="xs:string">
                    <xs:annotation>
                      <xs:appinfo>
                        <b:fieldInfo justification="left" pos_offset="0" pos_length="6" sequence_number="6" />
                      </xs:appinfo>
                    </xs:annotation>
                  </xs:element>
                  <xs:element name="D7" type="xs:string">
                    <xs:annotation>
                      <xs:appinfo>
                        <b:fieldInfo justification="left" pos_offset="0" pos_length="19" sequence_number="7" />
                      </xs:appinfo>
                    </xs:annotation>
                  </xs:element>
                  <xs:element name="D8" type="xs:string">
                    <xs:annotation>
                      <xs:appinfo>
                        <b:fieldInfo justification="left" pos_offset="0" pos_length="7" sequence_number="8" />
                      </xs:appinfo>
                    </xs:annotation>
                  </xs:element>
                </xs:sequence>
              </xs:complexType>
            </xs:element>
          </xs:sequence>
        </xs:complexType>
      </xs:element>
    </xs:schema>
    Thursday, July 9, 2009 9:59 AM
  • any help how to do this
    Friday, July 10, 2009 6:55 AM
  • Hi

    Out of the box, you cannot debatch in a way such that all the records will have a header as well.

    In the flat file disassembler you specify the header schema, body schema and footer schema. The hader and footer schemas are used to determine what is the body, and only the body is then moved on.

    If you need data from the header, consider promoting them to MessageContextPropertyBase properties (that is a property of the elements in your property schema). This way they follow the single record as context. Then you can use demotion in the assembler component to get the values back into the message you need output.


    eliasen, representing himself and not the company he works for.
    Three times MVP and three times MCTS in BizTalk.
    Blog: http://blog.eliasen.dk
    Friday, July 10, 2009 8:17 AM
  • Is there a Preserve Header option on the flat file disassembler?  I think it preserves the message to the context, but can't remember if it does it for all the debatched messages.  Either way Eliasen is correct, it would be in the context.  I think you're header and record need to be separate schemas (so run the wizard twice). 

    Now the unexpected \r\n that's a different story.  You have a pipeline (.btp) to process your flat file have to tested it with pipeline.exe?  Also be sure your delimiter is correct, CR LF vs the CR used by UNIX format. 

    Hope that helps.

    -Dan
    Friday, July 10, 2009 4:13 PM
  • For the header you can select Preserve Header and it puts the header XML into a message context for each debatched message. Here is an example on how to use it:
    http://seroter.wordpress.com/2007/08/23/debatching-flat-files-into-xml-with-header-intact/


    Regards,
    Thiago Almeida - http://connectedthoughts.wordpress.com
    • Proposed as answer by eliasen Sunday, July 12, 2009 8:50 PM
    • Marked as answer by Andrew_ZhuModerator Thursday, July 16, 2009 1:29 AM
    Saturday, July 11, 2009 12:46 AM
    Answerer
  • hi  Almeida
    thanks for reply
    but in my requirement we do not have to use orchestration;this project is an publisher application so we have to submit data after debatching with header and detail record into message box;
    no use of orchestration;how i will get promoted value inside recieve pipeline itself;
    Monday, July 13, 2009 11:07 AM
  • You could still build the messages in the orchestration and then submit them back to the message box. But if you don't want to use an orchestration for performance or other reasons you will have to create your own custom pipeline component.

    Your custom pipeline component would inherit from the flat file disassembler component (FFDasmComp). On the first GetNext method you would call the base.GetNext method and get the header from the message context. Then construct the final message returned by GetNext by combining the header with the body every time GetNext is called.
    There's an explanation here on the flat file disassembler class:
    http://www.codeproject.com/KB/biztalk/DebatchingFlatfile.aspx

    You will have to read on writing custom disassembler pipeline components (many blog posts and other online entries about it).
    Thiago Almeida - http://connectedthoughts.wordpress.com
    Monday, July 13, 2009 8:04 PM
    Answerer
  • hi i am able to debatch file in pipeline with header and detail node ;but now when i am setting recoverable interchange property true then if any record is invalid its giving error; because only valid message should go further;Please suggest if i am doing any thing wrong
    Wednesday, July 15, 2009 12:34 PM
  • If you want the records with errors to fail and the others to go through then you are doing it right by enabling recoverable interchange. It was already a great feature of BizTalk 2006 and 2006 R2, and there are some enhancements in 2009 as Randal explains on this post:
    http://biztalkmessages.vansplunteren.net/2009/03/26/recoverable-interchange-processing-enhancements-in-biztalk-2009/

    Thiago Almeida - http://connectedthoughts.wordpress.com
    Thursday, July 16, 2009 12:56 AM
    Answerer