locked
Run Pipeline based on Flag Value RRS feed

  • Question

  • Hi,

    I have requirement where I need to run pipeline based on flag value.

    I have one table where the flag values are getting stored, if for any particular record flag value is 1, then pipeline should run else it should not.

    Please help me with the solution

    Thanks for your time :)


    Pankaj

    Thursday, November 15, 2018 1:18 AM

All replies

  • Hi Pankaj,

    You could use a Lookup activity to query the flag value stored in the table, then use an IF activity to check whether its value is 1, finally refer an Execute pipeline activity to IF-TRUE activity, the Execute pipeline activity will invoke the piepeline you want to run, see the below example:

    {
        "name": "pipeline28",
        "properties": {
            "activities": [
                {
                    "name": "Lookup1",
                    "type": "Lookup",
                    "policy": {
                        "timeout": "7.00:00:00",
                        "retry": 0,
                        "retryIntervalInSeconds": 30,
                        "secureOutput": false,
                        "secureInput": false
                    },
                    "typeProperties": {
                        "source": {
                            "type": "SqlSource"
                        },
                        "dataset": {
                            "referenceName": "AzureSqlTable1", // this table stores the flag value
                            "type": "DatasetReference"
                        }
                    }
                },
                {
                    "name": "If Condition1",
                    "type": "IfCondition",
                    "dependsOn": [
                        {
                            "activity": "Lookup1",
                            "dependencyConditions": [
                                "Succeeded"
                            ]
                        }
                    ],
                    "typeProperties": {
                        "expression": {
                            "value": "@equals(activity('Lookup1').output.firstRow.flag,'1')", // this is to check wether the flag value is '1'
                            "type": "Expression"
                        },
                        "ifTrueActivities": [
                            {
                                "name": "Execute Pipeline1",
                                "type": "ExecutePipeline",
                                "typeProperties": {
                                    "pipeline": {
                                        "referenceName": "pipeline1", // if check is true, the pipeline you want to run.
                                        "type": "PipelineReference"
                                    }
                                }
                            }
                        ]
                    }
                }
            ]
        }
    }

    Some useful link: 

    https://docs.microsoft.com/en-us/azure/data-factory/control-flow-lookup-activity

    https://docs.microsoft.com/en-us/azure/data-factory/control-flow-if-condition-activity

    https://docs.microsoft.com/en-us/azure/data-factory/control-flow-execute-pipeline-activity

    • Proposed as answer by Bhushan Gawale Thursday, November 15, 2018 6:29 AM
    Thursday, November 15, 2018 2:27 AM
  • Hi Wang,

    Thanks for your help,

    Facing one issue

    Complete Pipeline it is successfully running and executing the pipeline inside true condition, while checking the String value 

    but if we are Checking boolen value in if condition i.e 

     "value": "@equals(activity('Lookup1').output.firstRow.flag,'1')"

    then the activity is successfully running but the pipeline inside true condition is not executing.  


    Pankaj

    Thursday, November 15, 2018 7:40 AM
  • what's your boolean value? true/false or 0/1?

    If it's true/false. the expression should be "value": "@equals(activity('Lookup1').output.firstRow.flag,true)" or

    "value": "@equals(activity('Lookup1').output.firstRow.flag,bool(1))"

    if it's 0/1, then the expression is "value": "@equals(activity('Lookup1').output.firstRow.flag,1)"

    • Edited by Wang Zhang Thursday, November 15, 2018 8:35 AM
    Thursday, November 15, 2018 8:29 AM
  • Thanks a lot Wang

    It is working fine now,

    i tried before this, using true but it was not working.

    Anyway now i am able to achieve my end goal.

    Thanks for your time :)


    Pankaj

    Thursday, November 15, 2018 10:19 AM
  • Good to hear. Thanks. :)
    Thursday, November 15, 2018 1:07 PM