locked
How does the On-Premise connector recognises already processed files RRS feed

  • Question

  • Hi,

    I am using the On-Premise connector within my logic apps. I have wondered how the connector recognises that the file is already processed by the engine. Does it persist the information internally? 

    Scenario: 

    Let's say we dropped a file into a drop folder. Logic apps connector picked up the file. However, we havent written any step to archive/delete the file. When the logic app triggers again, it will not pick up the file. How is this done internally? There is no documentation on the microsoft site.


    Friday, August 31, 2018 8:17 AM

Answers

  • Hi Shankar,

    In context of OPDG, I would get started with this doc (be sure to read OPDG docs that link to and are on PowerBI docs as well) and carefully review this visual. So, connectors listen to messages from Azure Service Bus queue internally and begins to start the workflow. As far as how a trigger detects & knows a "change" has occurred goes, whether it's filesystem or database record change, either polling or push APIs are used. In your case, because it concerns a change in filesystem, particular connector you're using probably uses polling pattern factoring in things like last time it checked for changes, timestamps, number of files in the directory, and may be other meta data from the files etc..I found this doc on triggers very useful in how they work as well. Hope this was helpful :)
    Saturday, September 1, 2018 12:17 AM

All replies

  • Hi Shankar,

    In context of OPDG, I would get started with this doc (be sure to read OPDG docs that link to and are on PowerBI docs as well) and carefully review this visual. So, connectors listen to messages from Azure Service Bus queue internally and begins to start the workflow. As far as how a trigger detects & knows a "change" has occurred goes, whether it's filesystem or database record change, either polling or push APIs are used. In your case, because it concerns a change in filesystem, particular connector you're using probably uses polling pattern factoring in things like last time it checked for changes, timestamps, number of files in the directory, and may be other meta data from the files etc..I found this doc on triggers very useful in how they work as well. Hope this was helpful :)
    Saturday, September 1, 2018 12:17 AM
  • Hi,

    It is internal architecture of Azure Service Bus for Polling and Pushing data (refer Link#1) request between the On-Premise and Cloud.

    Connectors uses the Actions and Triggers which creates the query request on Azure Service Bus and On-Premise Data Gateway passes the required data to be processed on request.

    Link#1 https://docs.microsoft.com/en-us/power-bi/service-gateway-onprem

    HTH


    Hope this Helps!!!! Regards, Note: Please Mark As Answered if you satisfy with Reply.

    Saturday, September 1, 2018 10:39 AM