none
Scheduled Triggers not working on pipeline

    Question

  • Hi, 

    I have configured everything from dataset, pipeline and then triggered it off manually from the website to generate and view the output. It works. However, when i schedule the trigger and execute it, the output doesnt seems to be refreshing.

    I am trying to copy the data from azureSQL to Azure Lake Store.

    Idrees

    Friday, October 19, 2018 5:07 PM

All replies

  • Hi, 

    I have configured everything from dataset, pipeline and then triggered it off manually from the website to generate and view the output. It works. However, when i schedule the trigger and execute it, the output doesnt seems to be refreshing.

    I am trying to copy the data from azureSQL to Azure Lake Store.

    Idrees

    My code of scheduled trigger looks like this:

    {
    "name": "trigger3",
    "properties": {
    "runtimeState": "Started",
    "pipelines": [
    {
    "pipelineReference": {
    "referenceName": "AZ_PL_H_adfv2Test_1Day_LastIteration_EMPL",
    "type": "PipelineReference"
    },
    "parameters": {
    "windowStartTime": "@trigger().scheduledTime"
    }
    }
    ],
    "type": "ScheduleTrigger",
    "typeProperties": {
    "recurrence": {
    "frequency": "Minute",
    "interval": 2,
    "startTime": "2018-10-19T15:13:00Z",
    "timeZone": "UTC"
    }
    }
    }
    }

    Friday, October 19, 2018 5:10 PM
  • What do you mean by "the output doesn't seems to be refreshing"

    Has the pipeline been triggered or not? You could go to monitor tab to check if there is any new pipeline run. 

    If there pipeline has been triggered, but only the output is not right, please share a little bit more details about your scenario. Are you doing incremental copy? What do your pipeline and dataset json look like?



    Saturday, October 20, 2018 8:42 AM
  • Hi,

    I had similar issues with the scheduled trigger. It has several limitations over the tumbling window trigger.
    In this case, it may be due to the unsupported backfilling of windows.

    Please review the topic at 
    https://docs.microsoft.com/en-us/azure/data-factory/concepts-pipeline-execution-triggers#trigger-type-comparison

    And consider changing to tumbling window triggers. The only annoying thing being the one-many mapping of triggers to pipelines, so you would need to create individual triggers instead of sharing a scheduled trigger across multiple pipelines.

    Kind regards

    Lawrence

    Monday, November 5, 2018 8:49 AM