locked
Pipeline running 2 slices when deployed instead of one RRS feed

  • Question

  • I'm trying to configure a pipeline to execute for one data slice and then sit at a ready state. When I deploy the pipeline, it generates 2 slices and runs twice.  How do I get it to only generate one slice and execute once?

    My dataset and pipeline are configured for 15 minute intervals.  I set the activity window for 13 minutes (see below).  I've also tried 14 and 15 minute activity windows, but get the same results.

    When pipeline is deployed, 2 data slices are created. Based on the example below, the data slices are for 2:15 - 2:30 and 2:30 - 2:45.

    Pipeline activity window configuration:

           "start": "2017-02-01T02:18:00Z",
            "end": "2017-02-01T02:31:00Z",
            "isPaused": false,
            "hubName": "rkwusofcdfdev_hub",
            "pipelineMode": "Scheduled"
        }
    }

    Scheduler configuration for copy activity in pipeline:

                    "scheduler": {
                        "frequency": "Minute",
                        "interval": 15
                    },
                    "name": "CopyWellboreCountBlobToSql"
                }
            ],

    Dataset availability configuration (identical for both input and output):

            "availability": {
                "frequency": "Minute",
                "interval": 15
            },
            "external": false,
            "policy": {}
        }
    }



    • Edited by bbainssla Thursday, February 2, 2017 3:05 PM
    Thursday, February 2, 2017 3:02 PM

Answers

  • I solved this on my own.  After putting everything together for my post, I realized that the Data Slices are scheduled starting on the hour. For example 15 minute intervals result in data slices from 2:00-2:15, 2:15-2:30, 2:30-2:45, etc.  I was setting the pipeline activity window so that it crossed 2 of the data slice intervals, e.g. 2:10-2:25.  This caused the 2:00-2:15 and 2:15-2:30 data slices to both execute.  Once I conformed my activity window to match a data slice, only a single data slice was generated and executed.
    • Marked as answer by bbainssla Thursday, February 2, 2017 4:07 PM
    Thursday, February 2, 2017 4:07 PM