locked
How to execute an activity in a pipeline even one path from previous activities reaches it. RRS feed

  • Question

  • Hi Team,

    I have a pipeline setup for as set of activities in my Data factory. 

    Have two storedprocedure activities as my last activity. one for all my success paths and one more for my error path.

    The stored procedure activity at the end of my success path is getting triggered on all paths being success. Whereas when there is a failure, the storepocedure activity is not getting triggered. I know that it is waiting for all the path.

    Activity1----------->S------>Activity2->S->Storedpocedure (Success)

    |                                          |

    E ----->Append Variable <----E

                          |

                          |

              stored procedure (Error)

    The Append variable activity is not getting triggered because it is waiting for both Activity 1 and activity 2 to fail for setting the value and calling the stored procedure.

    But I want it to be called even if one of the activity fails.

    Please give some options.

    Tuesday, September 3, 2019 2:55 AM

Answers

  • Hello AshokBurra and thank you for your inquiry.  Please allow me to explain some examples.  If the following examples and explanations do not help, please upload a picture.  In these examples I use 'Set Variable' activities because they do not require any datasets, and I can control them directly.  Substitute your own activities in the place of the 'Set Variables'.

    Direct Dependencies:

    Here I have structured thee activities such that the third 'I will run' will execute even if 'I sometimes fail' fails or succeeds.  This is because I used the 'Completion' dependency (shown by blue line).  The completion dependency applies when the activity succeeds or fails.   This example is fairly straightforward, but what if there is another activity in the middle?  That is the next example.

    Branching dependencies:

    Here I have extended the previous example.  Now we have different actions to take, depending upon the status of activity 'I sometimes fail'.  Upon success there is activity 'on success', and upon failure, there is activity 'on fail'.  Activity 'I sometimes fail' either succeeds of fails, not both;  This means only one of the activities that comes after will run, not both.  This is where the dependency type 'Skipped' is useful (shown by grey line).   The 'skipped' dependency happens when an activity is not run.  By connecting 'on fail' to 'I will run' by both the dependency type 'skipped' and dependency type 'success', I cause execution to reach 'I will run' in either case.  However, if 'on fail' itself failed, execution would not reach 'I will run'.  Does this make sense?

    Tuesday, September 3, 2019 6:55 PM

All replies

  • Hello AshokBurra and thank you for your inquiry.  Please allow me to explain some examples.  If the following examples and explanations do not help, please upload a picture.  In these examples I use 'Set Variable' activities because they do not require any datasets, and I can control them directly.  Substitute your own activities in the place of the 'Set Variables'.

    Direct Dependencies:

    Here I have structured thee activities such that the third 'I will run' will execute even if 'I sometimes fail' fails or succeeds.  This is because I used the 'Completion' dependency (shown by blue line).  The completion dependency applies when the activity succeeds or fails.   This example is fairly straightforward, but what if there is another activity in the middle?  That is the next example.

    Branching dependencies:

    Here I have extended the previous example.  Now we have different actions to take, depending upon the status of activity 'I sometimes fail'.  Upon success there is activity 'on success', and upon failure, there is activity 'on fail'.  Activity 'I sometimes fail' either succeeds of fails, not both;  This means only one of the activities that comes after will run, not both.  This is where the dependency type 'Skipped' is useful (shown by grey line).   The 'skipped' dependency happens when an activity is not run.  By connecting 'on fail' to 'I will run' by both the dependency type 'skipped' and dependency type 'success', I cause execution to reach 'I will run' in either case.  However, if 'on fail' itself failed, execution would not reach 'I will run'.  Does this make sense?

    Tuesday, September 3, 2019 6:55 PM
  • Hi MartinJaffer,

    Thanks for your detailed explanation. Both completion and skipped activity slipped my attention. They both lost my thoughts as I was working very late in the night on a holiday.

    Your examples have triggered my thoughts to work. Will try and keep you posted.

    Once again thanks for taking time in explaining with examples.

    Regards

    Ashok.

    Wednesday, September 4, 2019 2:17 AM
  • Understandable.  There are some nuances that are easy to overlook.  I took the liberty of marking as answer given your positive response.
    Wednesday, September 4, 2019 5:28 PM
  • Hi MartinJaffer,

    Yes, your response was helpful. I was able to get things implemented with your suggestions. Thanks for it.

    I was unable to attach a picture of my completed solution as I get the portal complaining about my registration to be certified. Don't know how to get my id certified.

    Regards

    Ashok 


    Ashok Burra

    Friday, September 6, 2019 4:18 PM
  • For certifying your registration, have you checked you email spam box or signed up for email alerts?
    Friday, September 6, 2019 9:20 PM

  • I was unable to attach a picture of my completed solution as I get the portal complaining about my registration to be certified. Don't know how to get my id certified.

    You may comment on below thread and request to verify your account. You would be able to attach pictures then. 

    https://social.msdn.microsoft.com/Forums/en-US/dc4002e4-e3de-4b1e-9a97-3702387886cc/verify-account-42?forum=reportabug


    Cheers
    Vaibhav
    MCSA (SQL Server 2014)

    Monday, September 9, 2019 9:30 AM