locked
Tumbling Window endTime Parameter ADFv2 RRS feed

  • Question

  • I am using Data Factory v2 and am trying to access the endTime for a Tumbling Window trigger inside a pipeline copy activity.  I have a Copy that is using a query for the source.  Inside that query I am trying to select all day for an incremental time period that matches the tumbling window start and end.  So:

    select * from @{item().sourceTableName}
    WHERE 
    @{item().modifiedColumn} > '@{trigger().startTime}'
    AND
    @{item().modifiedColumn} <= '@{trigger().endTime}'

    The problem is I can't figure out how to access the trigger's window's end time.  trigger().endTime is not valid.  How can I or should I be doing this?

    Tuesday, February 27, 2018 7:54 PM

Answers

  • Hi, please reference the samples in this doc (https://docs.microsoft.com/en-us/azure/data-factory/how-to-create-tumbling-window-trigger).

    Pipeline can't directly use trigger's variables. To reference trigger's windowStartTime and windowEndTime in pipeline, you will need to define two parameters in pipeline, pass trigger().outputs.windowStartTime and trigger().outputs.windowEndTime to these two parameters, and then use the defined parameters in pipeline. 

    In this case, if you are using ADFV2 UI, I suggest you use the copy data wizard (the copy data button at the home page). Select "Run regularly on schedule" at the first page, and select "USE QUERY" at the select table page, the copy wizard will automatically generate the sample query with related parameters for you.

    • Marked as answer by AzureFrank Friday, March 16, 2018 5:42 PM
    Wednesday, February 28, 2018 3:36 AM

All replies

  • Hi, please reference the samples in this doc (https://docs.microsoft.com/en-us/azure/data-factory/how-to-create-tumbling-window-trigger).

    Pipeline can't directly use trigger's variables. To reference trigger's windowStartTime and windowEndTime in pipeline, you will need to define two parameters in pipeline, pass trigger().outputs.windowStartTime and trigger().outputs.windowEndTime to these two parameters, and then use the defined parameters in pipeline. 

    In this case, if you are using ADFV2 UI, I suggest you use the copy data wizard (the copy data button at the home page). Select "Run regularly on schedule" at the first page, and select "USE QUERY" at the select table page, the copy wizard will automatically generate the sample query with related parameters for you.

    • Marked as answer by AzureFrank Friday, March 16, 2018 5:42 PM
    Wednesday, February 28, 2018 3:36 AM
  • Thanks .
    Monday, August 13, 2018 7:31 AM