none
Aggregating/Consolidating multiple Files rows count into Single file-Design Approach

    Question

  • 15 different type flat files will be in Source Folder .BizTalk to pick up one by one from Source Folder and calculate the no.of rows count from each flat file. After that Corresponding File Name and Record Count to be added to Control file.

    Control file is a summary of all 15 files Names and corresponding Record Count. Finally All the source files along with Control file to be dropped into Target Folder (i.e.total 16 files).

     

    Sample Control File will be(comma separated):

    Ex: File1.dat,10

       File2.dat,0

    ……

    File15.dat,700

     

    Note:Non Uniform data in all 15 files

    Could you please suggest Best Solution Approach for above Scenario? Thanks In Advance



    • Edited by BizQ Sunday, December 25, 2016 9:01 AM
    Sunday, December 25, 2016 8:46 AM

All replies

  • Hi BizQ,

    Can you provide more information regarding your requirement.when this all 15 files comes to receive folder? Is there any dependency of all file to each other?

    without above information the solution will be not correct .

    But you can try below one in the mean time:

    Just implement sequential conay . Use Receive Port name for corelation .So now you can read file name and count the no of records in each file and can write in control file and put files in a temp location . Once you will receive your all 15 file then move all the 15 file and your control file from temp location to desired folder.

     Regards

    Abhay Giri

    Monday, December 26, 2016 6:39 AM
  • Hi BizQ,

    It is common scenario and can be achieved using sequential convoy pattern. Please create an orchestration using sequential convoy that uses correlation on the receive port name. As you receive the file in to the orchestration, write the filename to an orchestration variable and finally create the control file from the string variable content.

    Cheers


    JB

    Tuesday, January 10, 2017 3:04 AM
  • You can of course use an orchestration, but in this case i would go for SSIS.
    Monday, January 30, 2017 9:07 PM