locked
ADF Pipeline not running RRS feed

  • Question

  • My pipeline is not executing(running) i have set it to run every 15 Minutes.

    Source Data Set :

    {
        "name": "Tweetfile",
        "properties": {
            "published": false,
            "type": "AzureDataLakeStore",
            "linkedServiceName": "caaAzureDataLakeStoreLinkedService",
            "typeProperties": {
                "fileName": "tweet.csv",
                "folderPath": "/",
                "format": {
                    "type": "TextFormat",
                    "columnDelimiter": ","
                }
            },
            "availability": {
                "frequency": "Minute",
                "interval": 15
            },
            "external": true,
            "policy": {}
        }
    }
    Out Put Data Set
    {
        "name": "tweetanalysis",
        "properties": {
            "published": false,
            "type": "AzureDataLakeStore",
            "linkedServiceName": "caaAzureDataLakeStoreLinkedService",
            "typeProperties": {
                "folderPath": "tweetanalysis/output/"
            },
            "availability": {
                "frequency": "Minute",
                "interval": 15
            }
        }
    }

    Pipe Line

    {
        "name": "RunPythonCode",
        "properties": {
            "description": "This is a pipeline to execute python code",
            "activities": [
                {
                    "type": "DataLakeAnalyticsU-SQL",
                    "typeProperties": {
                        "scriptPath": "/scripts/USQL/Script.usql",
                        "scriptLinkedService": "caaAzureDataLakeStoreLinkedService",
                        "degreeOfParallelism": 1,
                        "priority": 100,
                        "parameters": {}
                    },
                    "inputs": [
                        {
                            "name": "Tweetfile"
                        }
                    ],
                    "outputs": [
                        {
                            "name": "tweetanalysis"
                        }
                    ],
                    "policy": {
                        "timeout": "06:00:00",
                        "concurrency": 1,
                        "executionPriorityOrder": "NewestFirst",
                        "retry": 1
                    },
                    "scheduler": {
                        "frequency": "Minute",
                        "interval": 15
                    },
                    "name": "PythonCodeExecution",
                    "linkedServiceName": "AzureDataLakeAnalyticsComputeService"
                }
            ],
            "start": "2015-08-08T00:00:00Z",
            "end": "2015-08-08T01:00:00Z",
            "isPaused": false,
            "hubName": "caaadf_hub",
            "pipelineMode": "Scheduled"
        }
    }

    Monday, December 18, 2017 5:25 PM

All replies

  • Although you have your schedule set for every 15 minutes, your start and end times are not within the current date range.  Try changing the part of your pipeline where it says this

            "start": "2015-08-08T00:00:00Z",
            "end": "2015-08-08T01:00:00Z",
            "isPaused": false,
            "hubName": "caaadf_hub",
            "pipelineMode": "Scheduled"

    and change it to this:

            "start": "2015-12-19T00:00:00Z",
            "end": "2015-12-25T01:00:00Z",
            "isPaused": false,
            "hubName": "caaadf_hub",
            "pipelineMode": "Scheduled"

    once you make this change, redeploy your pipeline.

    The basic idea is that your start and end times have to include the current date.  Also remeber that the times are zulu time, which, on the wast coast US I think is -5 hours.

    Hope this helps.


    • Proposed as answer by mschandler Friday, December 22, 2017 3:44 PM
    Thursday, December 21, 2017 2:16 AM