none
Azure Data factory - passing parameter to trigger

    Question

  • Hello.

    I am trying to figure out how to pass windowStartTime and endtime to pipeline from trigger.

    Here's json file:

    {
        "name": "trigger1",
        "properties": {
            "runtimeState": "Started",
            "pipeline": {
                "pipelineReference": {
                    "referenceName": "pipeline1",
                    "type": "PipelineReference"
                },
                "parameters": {
                    "windowStart": "@trigger().outputs.windowStartTime",
                    "windowEnd": "@trigger().outputs.windowEndTime"
                }
            },
            "type": "TumblingWindowTrigger",
            "typeProperties": {
                "frequency": "Minute",
                "interval": 15,
                "startTime": "2018-07-06T13:45:00.000Z",
                "delay": "00:00:00",
                "maxConcurrency": 50,
                "retryPolicy": {
                    "intervalInSeconds": 30
                }
            }
        }
    }

    and here's powershell script(doesn't work either way with or without params):

    $params = @"
    {
                    "windowStart": "@trigger().outputs.windowStartTime",
                    "windowEnd": "@trigger().outputs.windowEndTime"
    }
    "@
    Set-AzureRmDataFactoryV2Trigger -DataFactoryName $df.DataFactoryName -ResourceGroupName $rg.ResourceGroupName -DefinitionFile .\trigger1.json -Name trigger1 -PipelineVariable $params

    here's what it shows when uploaded:

    {
    "name": "trigger1",
    "properties": {
    "runtimeState": "Stopped",
    "pipeline": {
    "pipelineReference": {
    "referenceName": "pipeline1",
    "type": "PipelineReference"
    }
    },
    "type": "TumblingWindowTrigger",
    "typeProperties": {
    "frequency": "Minute",
    "interval": 15,
    "startTime": "2018-07-06T13:45:00Z",
    "delay": "00:00:00",
    "maxConcurrency": 50,
    "retryPolicy": {
    "intervalInSeconds": 30
    }
    }
    }
    }

    So for some reason that pipeline variable part doesn't get consumed, which is weird seeing that you can create a copy pipeline with that.

    Any help will be appreciated.

    Thanks,


    byungjoon yoon, software engineer at servicelink

    Friday, July 6, 2018 8:07 PM

Answers

  • Hi Byungjoon,

    Did you set parameter in your pipeline json? Use

    "parameters": {
    "windowStart": "@trigger().outputs.windowStartTime",
    "windowEnd": "@trigger().outputs.windowEndTime"
    }

    in trigger definition is for purpose of passing value to pipeline parameter. You'll need manually define parameter in your pipeline like:

    "parameters": {
    "windowStart": {
    "type": "String"
    },
    "windowEnd": {
    "type": "String"
    }

    }

    Please add the parameter definition in pipeline json and have another try.

    Also, we provide an intuitive UI tool for your scenario: https://docs.microsoft.com/en-us/azure/data-factory/how-to-create-schedule-trigger#data-factory-ui

    Thanks.

    Saturday, July 7, 2018 1:26 AM

All replies

  • Hi Byungjoon,

    Did you set parameter in your pipeline json? Use

    "parameters": {
    "windowStart": "@trigger().outputs.windowStartTime",
    "windowEnd": "@trigger().outputs.windowEndTime"
    }

    in trigger definition is for purpose of passing value to pipeline parameter. You'll need manually define parameter in your pipeline like:

    "parameters": {
    "windowStart": {
    "type": "String"
    },
    "windowEnd": {
    "type": "String"
    }

    }

    Please add the parameter definition in pipeline json and have another try.

    Also, we provide an intuitive UI tool for your scenario: https://docs.microsoft.com/en-us/azure/data-factory/how-to-create-schedule-trigger#data-factory-ui

    Thanks.

    Saturday, July 7, 2018 1:26 AM
  • my bad, apparently I named it windowStartTime in pipeline, and it was sending windowStart. 

    Changing that fixed the issue.

    Thanks,


    byungjoon yoon, software engineer at servicelink

    Monday, July 9, 2018 6:27 PM