none
Distinct values in SSIS RRS feed

  • Question

  •  

    Hi Team,

     

      TableA has (id, name,val)

      TableB has (id,access)

     

    I need id,name,val,access.... I have done a merge join between these tables so that, On inner join

    am getting the id,name,val,access ... Is it correct ?

     

    My problem is, I am getting lot of duplicates and I need to avoid those duplicates by providing distinct condition.

     

    How can we achieve this?

     

    I have tried sort task but, it will do a full blockage... Any help please?

     

    Regards,

    Venkatesan Prabu .J

    http://venkattechnicalblog.blogspot.com/

     

     

    Tuesday, August 26, 2008 1:03 PM

All replies

  • The aggregate can also group records, but it's fully blocking as well. It might be faster to stage the rows to a working table in SQL, then select them back out with a DISTINCT or GROUP BY.

     

    Also, if TableA and TableB are in the same database, perform the join and the DISTINCT in one SQL statement that's the source for the rest of the flow.

    Wednesday, August 27, 2008 2:33 AM
    Moderator
  • If both tables are in one database, you a sql source with group or distinct T-SQL

    command

    Wednesday, August 27, 2008 7:33 AM
  • I had the same issue but I am using RedShift. In order to get what I needed I would have needed to create a temp table and or working table in Redshift then perform another query to get the values. For consistence and performance I used the aggregate function and it was extremely quick and no need for extra tables to drop or create. Thanks for the great advice!
    Thursday, May 19, 2016 2:49 PM