locked
Rollback Failed Pipeline to avoid partial commit RRS feed

  • Question

  • Is there a way to rollback failed pipeline in Azure Data factory?

    Example - I have multiple pipelines and each pipeline have multiple data flow activities where some activities are making partial commit and some are failing for that pipeline. So how can i rollback complete pipeline?

    Please suggest...

    Friday, January 31, 2020 11:19 PM

Answers

  • Hello Sudehely and thank you for your question.  I assume you mean 'roll back data' rather than 'roll back changes to pipeline'.  Does one pipeline depend upon another pipeline?

    There is no 'rollback' activity, so you will have to make your own.  I have created a pipeline logic below with the following assumptions:

    • activities are squential;  if change1 succeeds, then do change2, if change1 and change2 succeed, do change3
    • a failed activity does not need to be rolled back, as it does not commit the change
    • rollbacks cannot fail
    • rollbacks must be made in sequence;  changes and rollbacks are a LIFO queue/stack
    • a pipeline where all changes succeed, does not need to be rolled back

    The above picture is hard to read, so please let me explain.  The 'Set variable' activities are stand-ins for the actual change and rollback activities.  The desired behavior is as follows:

    • If change1 succeeds, do change2
    • If change2 succeeds do change3
    • If change3 fails do rollback2 (and then do rollback1)
    • If rollback2 happens, then do rollback1
    • If change2 fails, do rollback1
    • If change1 fails, do nothing

    When dependencies from multiple activities point to a single activity, the conditions are AND'ed together.  When multiple dependencies from a single activity point to a single activity, they are OR'ed together.  The logic is accomplished by the below dependencies:

    • change1 -> change2 via success
    • change1 -> rollback1 via success
    • change2 -> change3 via succcess
    • change2 -> rollback1 via completion
    • change2 -> rollback2 via success
    • change3 -> rollback1 via failure & skipped
    • change3 -> rollback2 via failure
    • rollback2 -> rollback1 via success & skipped

    Below is the pipeline code for my example.  Replace the "variableName":"ok" with "variableName":"fail" to make an activity fail.

    {
        "name": "Rollback model",
        "properties": {
            "activities": [
                {
                    "name": "change1",
                    "type": "SetVariable",
                    "dependsOn": [],
                    "userProperties": [],
                    "typeProperties": {
                        "variableName": "ok",
                        "value": "ok"
                    }
                },
                {
                    "name": "change2",
                    "type": "SetVariable",
                    "dependsOn": [
                        {
                            "activity": "change1",
                            "dependencyConditions": [
                                "Succeeded"
                            ]
                        }
                    ],
                    "userProperties": [],
                    "typeProperties": {
                        "variableName": "ok",
                        "value": "ok"
                    }
                },
                {
                    "name": "change 3",
                    "type": "SetVariable",
                    "dependsOn": [
                        {
                            "activity": "change2",
                            "dependencyConditions": [
                                "Succeeded"
                            ]
                        }
                    ],
                    "userProperties": [],
                    "typeProperties": {
                        "variableName": "ok",
                        "value": "ok"
                    }
                },
                {
                    "name": "roll back 2",
                    "type": "SetVariable",
                    "dependsOn": [
                        {
                            "activity": "change 3",
                            "dependencyConditions": [
                                "Failed"
                            ]
                        },
                        {
                            "activity": "change2",
                            "dependencyConditions": [
                                "Succeeded"
                            ]
                        }
                    ],
                    "userProperties": [],
                    "typeProperties": {
                        "variableName": "ok",
                        "value": "ok"
                    }
                },
                {
                    "name": "roll back 1",
                    "type": "SetVariable",
                    "dependsOn": [
                        {
                            "activity": "change1",
                            "dependencyConditions": [
                                "Succeeded"
                            ]
                        },
                        {
                            "activity": "roll back 2",
                            "dependencyConditions": [
                                "Succeeded",
                                "Skipped"
                            ]
                        },
                        {
                            "activity": "change2",
                            "dependencyConditions": [
                                "Completed"
                            ]
                        },
                        {
                            "activity": "change 3",
                            "dependencyConditions": [
                                "Failed",
                                "Skipped"
                            ]
                        }
                    ],
                    "userProperties": [],
                    "typeProperties": {
                        "variableName": "ok",
                        "value": "ok"
                    }
                }
            ],
            "variables": {
                "fail": {
                    "type": "Boolean"
                },
                "ok": {
                    "type": "String"
                }
            },
            "annotations": []
        }
    }

    • Marked as answer by Sudehely Tuesday, February 4, 2020 3:04 PM
    Monday, February 3, 2020 7:27 PM

All replies

  • Hello Sudehely and thank you for your question.  I assume you mean 'roll back data' rather than 'roll back changes to pipeline'.  Does one pipeline depend upon another pipeline?

    There is no 'rollback' activity, so you will have to make your own.  I have created a pipeline logic below with the following assumptions:

    • activities are squential;  if change1 succeeds, then do change2, if change1 and change2 succeed, do change3
    • a failed activity does not need to be rolled back, as it does not commit the change
    • rollbacks cannot fail
    • rollbacks must be made in sequence;  changes and rollbacks are a LIFO queue/stack
    • a pipeline where all changes succeed, does not need to be rolled back

    The above picture is hard to read, so please let me explain.  The 'Set variable' activities are stand-ins for the actual change and rollback activities.  The desired behavior is as follows:

    • If change1 succeeds, do change2
    • If change2 succeeds do change3
    • If change3 fails do rollback2 (and then do rollback1)
    • If rollback2 happens, then do rollback1
    • If change2 fails, do rollback1
    • If change1 fails, do nothing

    When dependencies from multiple activities point to a single activity, the conditions are AND'ed together.  When multiple dependencies from a single activity point to a single activity, they are OR'ed together.  The logic is accomplished by the below dependencies:

    • change1 -> change2 via success
    • change1 -> rollback1 via success
    • change2 -> change3 via succcess
    • change2 -> rollback1 via completion
    • change2 -> rollback2 via success
    • change3 -> rollback1 via failure & skipped
    • change3 -> rollback2 via failure
    • rollback2 -> rollback1 via success & skipped

    Below is the pipeline code for my example.  Replace the "variableName":"ok" with "variableName":"fail" to make an activity fail.

    {
        "name": "Rollback model",
        "properties": {
            "activities": [
                {
                    "name": "change1",
                    "type": "SetVariable",
                    "dependsOn": [],
                    "userProperties": [],
                    "typeProperties": {
                        "variableName": "ok",
                        "value": "ok"
                    }
                },
                {
                    "name": "change2",
                    "type": "SetVariable",
                    "dependsOn": [
                        {
                            "activity": "change1",
                            "dependencyConditions": [
                                "Succeeded"
                            ]
                        }
                    ],
                    "userProperties": [],
                    "typeProperties": {
                        "variableName": "ok",
                        "value": "ok"
                    }
                },
                {
                    "name": "change 3",
                    "type": "SetVariable",
                    "dependsOn": [
                        {
                            "activity": "change2",
                            "dependencyConditions": [
                                "Succeeded"
                            ]
                        }
                    ],
                    "userProperties": [],
                    "typeProperties": {
                        "variableName": "ok",
                        "value": "ok"
                    }
                },
                {
                    "name": "roll back 2",
                    "type": "SetVariable",
                    "dependsOn": [
                        {
                            "activity": "change 3",
                            "dependencyConditions": [
                                "Failed"
                            ]
                        },
                        {
                            "activity": "change2",
                            "dependencyConditions": [
                                "Succeeded"
                            ]
                        }
                    ],
                    "userProperties": [],
                    "typeProperties": {
                        "variableName": "ok",
                        "value": "ok"
                    }
                },
                {
                    "name": "roll back 1",
                    "type": "SetVariable",
                    "dependsOn": [
                        {
                            "activity": "change1",
                            "dependencyConditions": [
                                "Succeeded"
                            ]
                        },
                        {
                            "activity": "roll back 2",
                            "dependencyConditions": [
                                "Succeeded",
                                "Skipped"
                            ]
                        },
                        {
                            "activity": "change2",
                            "dependencyConditions": [
                                "Completed"
                            ]
                        },
                        {
                            "activity": "change 3",
                            "dependencyConditions": [
                                "Failed",
                                "Skipped"
                            ]
                        }
                    ],
                    "userProperties": [],
                    "typeProperties": {
                        "variableName": "ok",
                        "value": "ok"
                    }
                }
            ],
            "variables": {
                "fail": {
                    "type": "Boolean"
                },
                "ok": {
                    "type": "String"
                }
            },
            "annotations": []
        }
    }

    • Marked as answer by Sudehely Tuesday, February 4, 2020 3:04 PM
    Monday, February 3, 2020 7:27 PM
  • Thanks Martin! this is great information and nice explanation!
    Tuesday, February 4, 2020 3:05 PM
  • Thank you for the positive feedback Sudehely.  This was an interesting challenge.
    Tuesday, February 4, 2020 8:12 PM