none
IF count of @rowset >= @threshold THEN

    Question

  • I am writing a cleanup script for invalid entries. I have written code to seperate a list of invalid entries from valid ones, but I need to compare the amount of invalid entries to a @threshold int value to prevent the script from running (and taking up resources) if there aren't enough to warrant the operation.

    I know I can use 'SELECT COUNT() AS count' to create a new rowset with an integer value inside of it, but I have no idea how to compare that with my scalar @threshold variable in an IF statement. What's the best way of doing this?

    Friday, September 7, 2018 9:17 PM

Answers

  • If you need to do conditional cleanup on some threshold. You need to breakup the process into 2 jobs

    • Job 1 - Get the count and store it in a file.
    • Job 2 - Now extract and verify if it satisfies the condition and clean up the data.

    To execute this pipeline, you can create a ADF job with one activity as job1 and next activity as job2.

    Please let me know if you need any clarification

     
    Tuesday, September 11, 2018 8:23 PM
    Moderator

All replies

  • Is @threshold value is retrieved from somewhere or Is that a value assigned in the U-SQL script?
    Friday, September 7, 2018 10:40 PM
    Moderator
  • Eventually this is going to be a procedure so it will be assigned when the proc is called in another script
    Friday, September 7, 2018 10:59 PM
  • If you need to do conditional cleanup on some threshold. You need to breakup the process into 2 jobs

    • Job 1 - Get the count and store it in a file.
    • Job 2 - Now extract and verify if it satisfies the condition and clean up the data.

    To execute this pipeline, you can create a ADF job with one activity as job1 and next activity as job2.

    Please let me know if you need any clarification

     
    Tuesday, September 11, 2018 8:23 PM
    Moderator