locked
Set-AzureRmDataFactoryV2Dataset Error Message: The document creation or update failed because of invalid reference RRS feed

  • Question

  • Scenario: I'm working on automation of ADFv2 deployment using PowerShell.  I have created the pipeline with one activity, one dataset, and one linked service using the portal.  The linked service has been tested and confirmed its working. 

    I'm creating a 2nd activity with a new dataset for which I have the JSON file and trying to deploy the dataset using Set-AzureRmDataFactoryV2Dataset.  I keep getting the failure below. Note that this JSON file is a copy/paste of the dataset we deployed via the UI - for testing purposes, I simply changed the path and the name. 

    Looking for guidance on how to resolve this:

    Command:
    Set-AzureRmDataFactoryV2Dataset -DataFactoryName MyADFV2Test -ResourceGroupName MyADFGroup -Name testDataset2 -DefinitionFile C:\Scratch\mmd_adf_v2_input1_adls.json -Force

    Full Error Message: 

    Set-AzureRmDataFactoryV2Dataset : HTTP Status Code: BadRequest
    Error Code: BadRequest
    Error Message: The document creation or update failed because of invalid reference 'myTestLinkedService'.

     

    Friday, January 24, 2020 2:17 AM

All replies

  • Hi MMD3D,

    Thanks for your query. We are looking into it and will provide an update once we have the findings ready.

    Thank you

    If a post helps to resolve your issue, please click the "Mark as Answer" of that post and/or click Answered "Vote as helpful" button of that post. By marking a post as Answered and/or Helpful, you help others find the answer faster.

    Monday, January 27, 2020 6:57 PM
  • Hi Hi MMD3D,

    Looks like you have created '
    myTestLinkedService' linked service, tested connection but haven't published it yet and trying to reference that linked service in the new dataset that you are trying to create using Powershell. 

    In order to reference any data factory entity from Powershell, please make sure those entities are published first. Please try publishing the linked service first from the portal and then try to run your Powershell script to create the new dataset/actvitiy.

    Hope this should resolve your error. If you still encounter any issues please do let us know.


    Thank you

    If a post helps to resolve your issue, please click the "Mark as Answer" of that post and/or click Answered"Vote as helpful" button of that post. By marking a post as Answered and/or Helpful, you help others find the answer faster.


    Monday, January 27, 2020 10:26 PM
  • Hi MMD3D,

    Just checking to see if the above answer was helpful to resolve your issue. If this answers your query, please do consider to click “Mark as Answer” and "Up-Vote" as it might be beneficial to other community members reading this thread. And, if you have any further query do let us know.

    Thank you

    If a post helps to resolve your issue, please click the "Mark as Answer" of that post and/or click Answered "Vote as helpful" button of that post. By marking a post as Answered and/or Helpful, you help others find the answer faster.

    Wednesday, January 29, 2020 7:28 AM
  • Hi MMD3D,

    Since we haven't heard back from you, we will assume you found your own resolution. If you found a different solution, would you please share it here with the community as it would benefit other with similar issue. If you still need assistance, do let us know.

    Thank you

    If a post helps to resolve your issue, please click the "Mark as Answer" of that post and/or click Answered "Vote as helpful" button of that post. By marking a post as Answered and/or Helpful, you help others find the answer faster.

    Wednesday, January 29, 2020 7:55 PM
  • Hi Guys,

    I am facing same problem while I am trying to deploy ARM Template in Azure using c# code. Below is my Json snippet on which I am getting an error :

    

    I got the below error :

     "code""DeploymentFailed",
      "message""At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.",
      "details": [
        {
          "code""BadRequest",
          "message""{\r\n  \"error\": {\r\n    \"code\": \"BadRequest\",\r\n    \"message\": \"The document creation or update failed because of invalid reference 'cdmprocessbatchconclude'

    Could you guys please help me to get it resolved? I am not getting that what I am doing wrong here


    Ramandeep

    Sunday, February 9, 2020 6:27 PM
  • Hi Ramandeep123,

    Thanks for reaching out. Could you please make sure 'cdmprocessbatchconclude' is published before you reference it? 

    Looking at the error message you are  referencing 'cdmprocessbatchconclude', in order to reference any data factory entity, it should published first before referencing. 

    Thank you

    If a post helps to resolve your issue, please click the "Mark as Answer" of that post and/or click Answered"Vote as helpful" button of that post. By marking a post as Answered and/or Helpful, you help others find the answer faster.


    Monday, February 10, 2020 11:41 PM
  • Hi Ramandeep123,

    Just checking to see if the above information was helpful to resolve the issue? If you the issue still persists, do let us know. 


    Thank you

    If a post helps to resolve your issue, please click the "Mark as Answer" of that post and/or click Answered "Vote as helpful" button of that post. By marking a post as Answered and/or Helpful, you help others find the answer faster.

    Thursday, February 13, 2020 3:06 AM