none
HOW TO SEND SPECIFIC RECORDS IN DATA FLOW RRS feed

  • Question

  •  

    hello, i have 100,000 records in my table. Out of that what means to me is only 500 records that I want to pass thru and send it to destination.

     

    The data I am trying to transport has a column ID with Not Null values in it.

     

    Please let me know how can we do it,

     

    Can we use a look up for ID from the other tables to just pass thru when it finds an ID match,

     

    Am i correct.

     

    Monday, April 21, 2008 3:47 PM

Answers

  • One way: Use a WHERE condition in your OLE DB Source component.  You should be using SQL already anyway, now just add a WHERE condition to grab just the records you need.

     

    Another way: Use a conditional split to direct your NULL valued rows to one path and your NON-NULL valued rows to another path.  If you don't want to use the NULL rows, go ahead and just let them die at the Conditional Split.

     

    Monday, April 21, 2008 3:52 PM
    Moderator

All replies

  • One way: Use a WHERE condition in your OLE DB Source component.  You should be using SQL already anyway, now just add a WHERE condition to grab just the records you need.

     

    Another way: Use a conditional split to direct your NULL valued rows to one path and your NON-NULL valued rows to another path.  If you don't want to use the NULL rows, go ahead and just let them die at the Conditional Split.

     

    Monday, April 21, 2008 3:52 PM
    Moderator
  • If both source and destination tables are on the same database, and you don't need to do any transformations, just use an Execute SQL Task to do something like:

     

    Code Snippet
    INSERT INTO destination (keyCol1, col2) SELECT keyCol1, col2 FROM source src WHERE NOT EXISTS (SELECT * FROM destination dest WHERE src.keyCol1 = dest.keyCol1)

     

     

    Also, please don't use all-caps in your post title. We will respond to sentence case as well =)

    Monday, April 21, 2008 3:53 PM