none
Azure data factory's Event trigger for pipeline not working for Blob Creation/ deletion most of the time

    Question

  • Hi,

    As Microsoft recently added Blob creation event triggers for ADF pipeline run, I was trying to make it work .

    According to the documentation, in Blob starts with we need to enter the container name  in the format "/containername/"

     I gave the container name  in /containername/ format  and even the full path  /containername/blobs/folder/file.csv    still the trigger is not starting my ADF pipeline.

    My containername is : ingestionset   so I give /ingestionset/  in blobPathBeginsWith. but no use

    Am I missing something here?

    Tuesday, June 26, 2018 6:39 PM

All replies

  • Did you create your trigger in UI? As the document also said only version 2 storage will be supported. Could you confirm your storage version? UI will help list only version 2 storage.
    Wednesday, June 27, 2018 1:59 PM
  • I created my trigger in the UI (actually I recreated it today after seeing there has been an issue that has been fixed). It is not working.

    When i edit the trigger the subscription is blank, so I have to select it again, which empties the storage account, so i have to select that again too. Not sure if it is related.

    (this is V2, northeurope)

    Friday, June 29, 2018 12:07 PM
  • As others have stated please make sure that the storage account is V2.

    Also make sure that the name of your trigger only contains alphanumeric letters, digits, and the '-' character. Whitespace or special characters are not allowed, we need to update the UX to enforce this.

    Finally, make sure that you have registered your subscription with the Event Grid and Data Factory resource providers (re-register if already registered please). Here is a link on how to do that: https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-manager-supported-services#portal



    Friday, June 29, 2018 3:51 PM
  • I can concur that for v2 the event trigger is currently not working.  Region: East US.
    Tuesday, July 3, 2018 1:55 PM
  • HiI have the same problem. Also after registering with the relevant resource providers. Is this an issue Microsoft is trying to fix? Is there any way to debug a trigger, to understand why it does not fire?
    Thursday, July 12, 2018 8:29 AM
  • Could you contact ADF support team if you have confirmed all the conditions listed by cesher?
    Thursday, July 12, 2018 8:40 AM
  • Hi, It works. I just need to register the microsoft.eventgrid inside the respective subscription and the remaining things as per the tutorial.
    Thursday, July 12, 2018 9:28 AM
  • Great to hear.
    Thursday, July 12, 2018 1:26 PM
  • Hi Fang,

    Thank you kindly for your help.  I have what is perhaps a silly question, but nonetheless important.

    If we define are trigger as @triggerBody().folderPath, does this mean that if a folder is uploaded that contains say 100 files that the pipeline would be triggered 100 times? I am afraid of this happening if this is the case.

    I am looking for just one trigger when uploading to the defined folder path.

    Thanks again and all the best!

    Thursday, July 12, 2018 3:44 PM
  • How many times the trigger fires depends on how the trigger is configured. If the trigger is looking for any blob inside a container or folder, then when 100 files are uploaded to that container or folder 100 events will be emitted.

    On the other hand if the trigger is configured to fire for a specific type of file say 'control.txt', which is part of the 100 files, then when the folder is uploaded a single event will be emitted. Then using the @triggerBody().folderPath property, which you should map to a pipeline parameter to utilize its value, you will have the path to the container/folder.

    Just as an aside, we are considering implementing some sort of batching mechanism if there is enough demand for one. But at the time, we do not have concrete plans to share in that regard.

    Friday, July 13, 2018 5:51 PM