none
Azure data factory's Event trigger for pipeline not consistently working for Blob Creation

    Question

  • I have 4 data factories, each one have one triggers listening to different blob containers. All blob containers are hosted in the same storage account. Until yesterday, the triggers used to work pretty reliably.

    However, today only one of the 4 triggers does trigger when a new file is created. Others triggers are never firing.

    I've tried re-creating the DF from scratch, recreating the blob containers, to no avail.

    I started getting random errors when publishing my changes, this for example:


    After the error, just publishing again goes through without error warning, but still the triggers do not trigger.

    How do I go and debug this?

    Wednesday, August 1, 2018 11:48 AM

All replies

  • There seems to be a bug with Blob storage triggers, if you have more than one trigger allocated to the same blob container, none of the triggers will fire.

    For some reasons (another bug, but this time in Data factories?), if you edit several times your trigger in the data factories windows, the data factory seems to loose track of the triggers it creates, and your single trigger may end up creating multiple duplicate triggers on the blob storage. This condition activates the first bug discussed above: the blob storage trigger doesn't trigger anymore.

    To fix this, delete the duplicate triggers. For that, navigate to your blob storage ressource in the Azure portal. Go to the Events blade. From there you'll see all the triggers that the data factories added to your blob storage. Delete the duplicates.

    In the picture above, the last 3 triggers are duplicates, created by the data factory. After I have removed 2 of them, then the event starts firing.

    Thursday, August 2, 2018 5:07 AM