none
How to deploy only selected pipelines from DataFactory? RRS feed

  • Question

  • Hello Techie!

    I wanted to deploy only selected pipelines from one environment to other. Currently i'm using ARM template to move all pipelines to other environments. Is there a way available to move only selected pipelines?

    Thanks

    Sunday, October 20, 2019 9:42 AM

Answers

  • Hi Santhosh,

    There's no in-house way of selecting pipelines to be exported or imported using ARM templates. There's a feedback on the feedback forum. 

    I would recommend you to upvote the feedback at the feedback forum. All the feedback you share, is closely monitored by the Data Factory Product team and implemented in future releases.

    However to get you unblocked now, I can suggest a workaround that you might want to try. In the arm_template.json file, under resources, look for the pipelines you want to remove/filter out. You can remove the objects for these pipelines and then deploy the json file in another environment.

    Hope this helps.

    Monday, October 21, 2019 9:08 AM
    Owner

All replies

  • You can think of implementing CICD process to deploy pipelines using Azure DevOps. 

    It will release only incremental changes i.e. Pipeline, triggers available in code repository and not available in target ADF and deploy them.

    Continuous integration and delivery (CI/CD) in Azure Data Factory

    If this is one time pipeline creation in target ADF, you can copy past the json code in target ADF. 

    Another option is to save the pipeline as template and export to local folder and in Target ADF, import this template. 


    Regards,
    Vaibhav

    Sunday, October 20, 2019 11:22 AM
  • As CICD is not enabled in my project. We can't go for this option. 

    With "Pipeline as template" option, we can go for one or two pipelines, in our case at times we need to deploy 5-10 pipelines out of 30+ Any other option available?

    Sunday, October 20, 2019 2:12 PM
  • Hi Santhosh,

    There's no in-house way of selecting pipelines to be exported or imported using ARM templates. There's a feedback on the feedback forum. 

    I would recommend you to upvote the feedback at the feedback forum. All the feedback you share, is closely monitored by the Data Factory Product team and implemented in future releases.

    However to get you unblocked now, I can suggest a workaround that you might want to try. In the arm_template.json file, under resources, look for the pipelines you want to remove/filter out. You can remove the objects for these pipelines and then deploy the json file in another environment.

    Hope this helps.

    Monday, October 21, 2019 9:08 AM
    Owner
  • As CICD is not enabled in my project. We can't go for this option. 

    With "Pipeline as template" option, we can go for one or two pipelines, in our case at times we need to deploy 5-10 pipelines out of 30+ Any other option available?

    What I do is, simply copy the Json code behind the Pipeline and paste it in Target ADF pipeline. Later manually fill in the Linked services and datasets. 

    Unfortunately no any quick solution available right now.


    Regards,
    Vaibhav

    Monday, October 21, 2019 11:31 AM
  • Thanks Chirag!

    Thought to use this approach. And will do up-vote for this feature.

    Tuesday, October 22, 2019 2:48 AM
  • Yes Vaibhav. Thanks for your suggestion.
    Tuesday, October 22, 2019 2:49 AM