locked
ADF pipeline execution failed suddenly. RRS feed

  • Question

  • We have couple of ADF pipelines running each day. Suddenly its start failing with below error messages. 

    Copy failed with error: 'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Failed to get the secret from key vault, secretName: ukfilesharepassword, secretVersion: , vaultBaseUrl: https://<<my key vault URL>>. The error message is: An error occurred while sending the request. The remote name could not be resolved: 'https://<<my key vault URL>>'.,Source=Microsoft.DataTransfer.ClientLibrary,StackTrace= at Microsoft.DataTransfer.Runtime.AzureKeyVaultHelper.<GetSecret>d__3.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
    at Microsoft.DataTransfer.Runtime.ConnectorHelper.<>c__DisplayClass110_3.<<CheckAndResolveAkvSecret>b__1>d.MoveNext(),'

    My keyvault URL is also working fine when I ping it. After few hours(5-6 hours), its automatically resolved and pipelines were running as normal.

    Any help would be appreciated. 

     

    • Edited by kkpatel97 Tuesday, April 28, 2020 4:12 AM
    Tuesday, April 28, 2020 4:10 AM

All replies

  • Hi there,

    Hi there,

    Has this happened multiple times? It could be a transient issue. 

    Can you please make sure that you have the following things done :

    • Please follow the steps that is linked in the doc to create a managed identity for your azure data factory? https://docs.microsoft.com/en-us/azure/data-factory/data-factory-service-identity

      Below powershell script can be used to associate the MSI. Using the new Azure Powershell Az Module :

      https://docs.microsoft.com/en-us/powershell/azure/new-azureps-module-az?view=azps-2.8.0
    PS C:\WINDOWS\system32> Set-AzDataFactoryV2 -ResourceGroupName <resourceGroupName> -Name <dataFactoryName> -Location <region>
    
    DataFactoryName   : ADFV2DemoFactory
    DataFactoryId     : /subscriptions/<subsID>/resourceGroups/<resourceGroupName>/providers/Microsoft.DataFactory/factories/ADFV2DemoFactory
    ResourceGroupName : <resourceGroupName>
    Location          : East US
    Tags              : {}
    Identity          : Microsoft.Azure.Management.DataFactory.Models.FactoryIdentity
    ProvisioningState : Succeeded

    Ref - https://stackoverflow.com/a/60460668/10653466

    • Make sure that the connection to the Azure Key Vault works by hitting "Test Connection" in the Linked Service.

    Hope this helps.

    Tuesday, April 28, 2020 9:13 AM
  • Thank for your suggestion. 

    During this error, when I check key vault by Test connection its getting succeeded.

    Also, I have checked Manager identity in my ADF property. there are 3 identity listed. 

    Managed Identity Object ID
    Managed Identity Tenant
    Managed Identity Application ID

    Do you still want me to create new?


    Thanks, Kalpesh Patel

    Wednesday, April 29, 2020 4:24 AM
  • Hi Kalpesh,

    Sorry for the delayed response. Is the issue resolved or do you still need assistance?

    Tuesday, May 5, 2020 4:58 AM