locked
Dynamically load flat files RRS feed

  • Question

  • how can we load multiple csv files under one folder into database.  The files can be csv or xls format.
    Thursday, April 15, 2010 2:53 AM

Answers

  • Hi Suresh,

    I would suggest you use 2 For Each loop one for *.csv and other for *.xls(I assume the structure of the 2 types of files are the same.)

    Now use a DFT inside each loop. The Foreach loop configured for *.csv u can use Flat file source and OLEDB Destination.

    In the other DFT use Excel Source and OLEDB destination.


    Sudeep's Domain
    Thursday, April 15, 2010 6:07 AM

All replies

  • In case all the csv files are similar (name and number of columns) then you can use foreach loop in combination with a data flow task to load the data to a database table. In case the files are not similar, then you have to use one data flow task for each csv file.

     


    Nitesh Rai- Please mark the post as answered if it answers your question
    Thursday, April 15, 2010 3:59 AM
  • Hi Suresh,

    I would suggest you use 2 For Each loop one for *.csv and other for *.xls(I assume the structure of the 2 types of files are the same.)

    Now use a DFT inside each loop. The Foreach loop configured for *.csv u can use Flat file source and OLEDB Destination.

    In the other DFT use Excel Source and OLEDB destination.


    Sudeep's Domain
    Thursday, April 15, 2010 6:07 AM
  • Hi Suresh,

    I have 11 csv file in different structure to load into 11 tables. So, do i need to use 11 DataFlow Task (Flat File Source -> OLE DB Destination) ?

     

    thanks

    Gubs

    Thursday, April 15, 2010 11:49 AM
  • Yes...
    Nitesh Rai- Please mark the post as answered if it answers your question
    • Proposed as answer by gubs Thursday, April 15, 2010 12:02 PM
    Thursday, April 15, 2010 11:52 AM
  • Hi Suresh,

    I have 11 csv file in different structure to load into 11 tables. So, do i need to use 11 DataFlow Task (Flat File Source -> OLE DB Destination) ?

     

    thanks

    Gubs

    If you can use third-party tools, check the commercial CozyRoc Data Flow Task Plus . It includes support for dynamic data flows where you can define only one data flow and have your source and destination have dynamic columns. You can also specify a mapping dictionary between the source and destination to guide the mappings.

    SSIS Tasks Components Scripts Services | http://www.cozyroc.com/
    Thursday, April 15, 2010 1:40 PM