locked
ADF event-based trigger starts working with delay RRS feed

  • Question

  • Hi All

    We have faced with the issue when event-based trigger in Azure Data Factory doesn't start work when we expect.

    We have a container/folder in blob storage where every night Azure microservice puts around 400 files. All files arrive during 20-30 secs. There is a trigger in Data Factory which starts work once new files have been created in blob storage. Trigger has a parameter like @triggerBody().FileName and we expect that around 400 triggers will be started immediately, or at least with delay 20-30 secs based on some files were come earlier, and some in 20-30 secs. But in fact, we have a situation when some triggers (around 260) start working immediately, then a pause period for 1 hour, next after pause around 110 triggers start work, then again a pause but for around 3 hours, and after that all remaining triggers (around 30) start work.

    So the schema looks like, all numbers and timing are approximately (our 400 files come at 1.30 AM UTC+3):
    first triggers: 1.30AM - 1.49 AM (approximately 260 triggers or accordingly 260 files are being processed)
    next scope: 2.42 AM - 2.49 AM (≈ 110 triggers)
    final scope: 5.42 AM - 5.43 AM (≈ 30 triggers)

    Could you try to help us and describe why it's going so? Probably, there are any limitations in ADF and not all files (and triggers based on them) can be handled immediately. But it's really strange to see delays such that, especially for 3 hours. In fact, file was uploaded into blob storage, but the event-based trigger starts work only 3 hours after that.

    Thanks in advance!


    • Edited by Artem_A Monday, May 11, 2020 4:08 PM
    Monday, May 11, 2020 3:58 PM

All replies

  • Thanks Artem ,

    At this time we are not unable to repro the issue . 

    For a deeper investigation and immediate assistance on this issue, if you have a support plan you may file a support ticket, else could you please send an email to azcommunity@microsoft.com with the below details, so that we can create a one-time-free support ticket for you to work closely on this matter. 
    Thread URL:
    Subscription ID: 
    Subject : Attn Himanshu
    Please let me know once you have done the same


    Thanks Himanshu
    ================
    If you think my answer helped you , please click "mark as answer " , this will be help other community members to get to the resolution faster
    Alone, we can do so little; together, we can do so much

    Thursday, May 14, 2020 1:25 AM
  • Hi Himanshu,

    An email with details was sent to the address you provided. Thanks for your helping!

    Thursday, May 14, 2020 8:51 AM
  • Hello Artem, 

    We did replied back and wanted to know if the issue was resolved .Can you please share the SR# with us , so that we can also follow up ?


    Thanks Himanshu
    ================
    If you think my answer helped you , please click "mark as answer " , this will be help other community members to get to the resolution faster
    Alone, we can do so little; together, we can do so much

    Thursday, May 28, 2020 10:24 PM
  • Hi Himanshu,

    Sorry for the delay with the answer. We got assistance from your side (SR 120051922002985) with detailed information on how this process is organized now and also that it will be improving for several weeks, so we are very glad about the support.

    Thanks once again for your assistance!

    Tuesday, June 9, 2020 8:21 AM