Need to run Pipeline based on Audit table Data Entry RRS feed

  • Question

  • Hello,

    I am coming from SSIS and SQL jobs background to run packages.

    My current work assignment is to Shift and Lift SSIS packages in ADF V2.

    Now in SSIS package , Any Data error comes we are ignoring failures  but in my Audit table I have maintaining PAckage status=Failed

    Now my requirement is that If Audit table is having Failed entry It will not move to next step.So in SQL job I know how to set this thing.

    But in adf how to I put such condition like , If Audit table is having failed entry send email to Stakeholder else IT will move to next step and execute that package.
    Friday, May 10, 2019 5:05 AM

All replies

  • Hi Aminesh,

    One way to do it is by using a logic app (please read this to learn about SQL connector for triggering logic apps) or an azure function(please read this to learn how to create a SQL based trigger for Azure function) in conjunction with ADF V2. 

    Once there's a change in the Audit table (row added), you can trigger an ADF V2 pipeline run by invoking it's REST API. To read more about it, please refer this doc. In the pipeline, you can have a "Get Metadata" Activity to read the firstRow of the Audit table and based on that, trigger a copy(or other) activity.

    Hope this helps.

    Monday, May 13, 2019 12:06 PM