locked
Cannot start trigger Azure Data Factory from PowerShell RRS feed

  • Question

  • Hi,

    I am releasing AzureDataFactory from ARM template through:

    1) Visual Studio 2017 in AzureResourceGroup project

    or

    2) Portal Azure > create resource >Template deployment (deploy using custom templates)

    with ARM

    {
      "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
      "contentVersion": "1.0.0.0",
      "parameters": {
        "location": {
          "type": "string",
          "allowedValues": [
            "westeurope"
          ]
        }
      },
      "variables": {
        "prefix": "[concat('test')]",
        "factoryName": "[concat(variables('prefix'),'-datafactory-6')]",
        "factoryId": "[concat('Microsoft.DataFactory/factories/', variables('factoryName'))]"
      },
      "resources": [
        {
          "name": "[variables('factoryName')]",
          "apiVersion": "2018-06-01",
          "type": "Microsoft.DataFactory/factories",
          "location": "[parameters('location')]",
          "identity": {
            "type": "SystemAssigned"
          }
        },
        {
          "name": "[concat(variables('factoryName'), '/Trigger Import Pipeline')]",
          "type": "Microsoft.DataFactory/factories/pipelines",
          "apiVersion": "2018-06-01",
          "properties": {
            "activities": [
              {
                "name": "Set Timestamp",
                "type": "SetVariable",
                "dependsOn": [],
                "userProperties": [],
                "typeProperties": {
                  "variableName": "Timestamp",
                  "value": {
                    "value": "@pipeline().TriggerTime",
                    "type": "Expression"
                  }
                }
              },
              {
                "name": "Set Folder Name",
                "type": "SetVariable",
                "dependsOn": [
                  {
                    "activity": "Set Timestamp",
                    "dependencyConditions": [
                      "Succeeded"
                    ]
                  }
                ],
                "userProperties": [],
                "typeProperties": {
                  "variableName": "FolderName",
                  "value": {
                    "value": "@concat(formatDateTime(variables('Timestamp'),'yyyyMMddHHmmss'))",
                    "type": "Expression"
                  }
                }
              }
            ],
            "variables": {
              "Timestamp": {
                "type": "String"
              },
              "FolderName": {
                "type": "String"
              }
            },
            "annotations": []
          },
          "dependsOn": [
            "[variables('factoryId')]"
          ]
        },
        {
          "name": "[concat(variables('factoryName'), '/ScheduleTrigger')]",
          "type": "Microsoft.DataFactory/factories/triggers",
          "apiVersion": "2018-06-01",
          "properties": {
            "annotations": [],
            "runtimeState": "Stopped",
            "pipeline": {
              "pipelineReference": {
                "referenceName": "Trigger Import Pipeline",
                "type": "PipelineReference"
              },
              "parameters": {}
            },
            "type": "ScheduleTrigger",
            "typeProperties": {
              "recurrence": {
                "frequency": "Hour",
                "interval": 2,
                "startTime": "2019-06-24T10:18:00.000Z",
                "timeZone": "UTC"
              }
            }
          },
          "dependsOn": [
            "[concat(variables('factoryId'), '/pipelines/Trigger Import Pipeline')]"
          ]
        }
      ],
      "outputs": {
        "DataFactory_Name": {
          "value": "[variables('factoryName')]",
          "type": "string"
        }
      }
    }

    It is deployed successful.

    Then I am enabling Trigger ScheduleTrigger with PowerShell execution (from local PC or DevOps pipeline)

    #
    # Start-DataFactoryV2Trigger.ps1
    #
    #param(
    #    [parameter(Mandatory = $true)] [string]$ResourceGroupName,
    #    [parameter(Mandatory = $true)] [string]$DataFactoryName
    #)
    
    $ResourceGroupName = "test-resource-datafactories"
    $DataFactoryName = "test-datafactory-6"
    
    Write-Host "ResourceGroupName     : `"$ResourceGroupName`""
    Write-Host "DataFactoryName       : `"$DataFactoryName`""
    
    #Login-AzureRmAccount 
    # set execution context
    #$subscription = (Get-AzureRmContext).Subscription
    $subscription = 'my-subscription'
    Select-AzureRmSubscription -SubscriptionId $subscription
    
    if ([string]::IsNullOrEmpty($(Get-AzureRmContext).Account)) {
        Add-AzureRmAccount
    }
    
    $ADF_Triggers = Get-AzureRmDataFactoryV2Trigger -ResourceGroupName $ResourceGroupName -DataFactoryName $DataFactoryName -ErrorVariable notPresent -ErrorAction SilentlyContinue
    Write-Host $ADF_Triggers.Name       
    if ($notPresent) {
        Write-Host "Trigger does not exist. Nothing to enable!"
    }
    else {
        $ADF_Triggers | ForEach-Object { 
            Write-Host "Enabling Pipeline Trigger $($_.name)"
            Start-AzureRmDataFactoryV2Trigger -ResourceGroupName $ResourceGroupName -DataFactoryName $DataFactoryName -Name $_.name -Force 
        }
    }

    And the error is:

    XXXXXXXXXXXXXXXXXXXXXXXXXXX

    Start-AzureRmDataFactoryV2Trigger : HTTP Status Code: BadRequest
    Error Code: BadRequest
    Error Message: Missing or invalid pipeline references for trigger ScheduleTrigger
    Request Id: f8416f6a-5c94-4d90-a41f-c61a6babdf5d
    Timestamp (Utc):07/18/2019 06:21:14
    At C:\Users\leko\source\repos\eBusiness\Integrations\src\Dgs.Ebiz.Integrations.AzureResourceGroup\Scripts\Start-DataFactoryV2Trigger.ps1:33 char:9
    +         Start-AzureRmDataFactoryV2Trigger -ResourceGroupName $Resourc ...
    +         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : CloseError: (:) [Start-AzureRmDataFactoryV2Trigger], ErrorResponseException
        + FullyQualifiedErrorId : Microsoft.Azure.Commands.DataFactoryV2.StartAzureDataFactoryTriggerCommand

    XXXXXXXXXXXXXXXXXXXXXXXXXXX

    But when I do manually to https://adf.azure.com/ portal and change something, for example trigger description, and then publish manully the factory. Run the PowerShell script and it's works.

    Can you please help me with this?


    Thursday, July 18, 2019 6:37 AM

Answers

  • The issue seems to be in the ARM template, the payload of trigger has a property called pipeline. For scheduled trigger it should be 'pipelines'. Refer the json of the trigger from the UI, and you will see that it should be an array of pipelines, not a single pipeline.
    Thursday, July 18, 2019 5:26 PM