Azure DevOps - Secure Copy the build file from Azure Artifactory to hosted env.

Question

we have CI CD in an hosted infra and we want to move CI to Azure DevOps/Pipelines and CD would be in our hosted infra within azure IAAS which is handled by Ansible

Once the build .jar file is generated and moved to azure artifacts, we have to copy the build file to our hosted infra in Azure IAAS

We are exploring couple of options for a secure copy 

1. SCP to a VM which is behind the firewall

2. SFTP to a VM which is behind the firewall

Do we have any other options to securely copy the build file from azure artifactory to hosted IAAS infra to continue the CD portion.

As part of PCI-DSS, we have to encrypt the data over transit and when we copy the data over internet, question is which is the best way to copy the file from azure artifact to the environment < iAAS>

Answer 1

Hello, Ravi, and thank you for using Azure!

I'm not sure I have much to add to your list as SFTP (SSH File Transfer Protocol) and SCP (Secure Copy) would be my two SSH recommendations, starting with SFTP.  I might add FTPS (FTP over SSL) as a consideration depending on your needs.  

The Azure Storage documentation on security might be worth checking out as well:

https://docs.microsoft.com/en-us/azure/storage/common/storage-security-guide#encryption-in-transit

Answer 2

Thanks for the suggestion. Still figuring out the best solution to copy the code from azure artifact over open internet to azure VM as data in transit to be encrypted.

Have asked the support on same question. still yet to hear a conclusive answer and the same solution should work in azure devops

Answer 3

Adding to this, AzCopy is encrypted by default and "Secure transfer required" would require encrypted connections to Azure Storage:

https://social.msdn.microsoft.com/Forums/en-US/b511ee4b-da7d-40b1-9de3-e4669135c7e8/azcopy-with-client-side-encryption?forum=windowsazuredata

"AzCopy connectivity is encrypted by default since it uses HTTPS, the "Secure transfer required" option enhances the security of your storage account by only allowing requests to the account from secure connections. For example, when you're calling REST APIs to access your storage account, you must connect by using HTTPS. "Secure transfer required" rejects requests that use HTTP. By default, the "Secure transfer required" option is disabled. You can turn on the "Secure transfer required" setting when you create a storage account in the Azure portal. You can also enable it for existing storage accounts."

Azure File Copy task is also worth mentioning:

https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-file-copy?view=azure-devops

"Use this task in a build or release pipeline to copy files to Microsoft Azure storage blobs or virtual machines (VMs).

The task is used to copy application files and other artifacts that are required in order to install the app; such as PowerShell scripts, PowerShell-DSC modules, and more.

When the target is Azure VMs, the files are first copied to an automatically generated Azure blob container and then downloaded into the VMs. The container is deleted after the files have been successfully copied to the VMs.

The task uses AzCopy, the command-line utility built for fast copying of data from and into Azure storage accounts."

Answer 4

Suggestion from MS DevOps team-

You could look at deployment groups, whereby an agent is installed on the Hosted Linux VM in Azure and that will pull the Azure DevOps published artifacts down to the machine. See https://docs.microsoft.com/en-us/azure/devops/pipelines/release/deployment-groups/?view=azure-devops.  That traffic goes over https. Here’s more on agent-to-Azure DevOps comm’s: https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/agents?view=azure-devops#communication-with-azure-pipelines.

We are trying this. Hope someone trying same thing helps.