locked
Parameterize Kusto linked services in Azure Data Factory RRS feed

  • Question

  • Hi,

    I am trying to create Kusto linkedservice dynamically, from the link https://docs.microsoft.com/en-us/azure/data-factory/parameterize-linked-services, look like it is possible to do it through code even if the UI is not supported yet.

    But I got the below errors in Practice.

    { "errorCode": "2054", "message": "Failed to decrypt sub-resource 'None' payload on cloud with error: Failed to decrypt sub-resource payload {\r\n \"properties\": {\r\n \"parameters\": {\r\n \"ServerName\": {\r\n \"type\": \"String\"\r\n }\r\n },\r\n \"annotations\": [],\r\n \"type\": \"AzureDataExplorer\",\r\n \"typeProperties\": {\r\n \"endpoint\": \"********************\",\r\n \"tenant\": \"********************\",\r\n \"servicePrincipalId\": \"********************\",\r\n \"database\": \"********************\",\r\n \"encryptedCredential\": \"********************\"\r\n }\r\n },\r\n \"name\": \"KustoServers1a6d75e913384faf8f7c421824cc9340\"\r\n} and error is: Failed to validate the signature because the content is tampered, the expect context is '{\"servicePrincipalId\":\"71acf48e-a842-4d99-8be3-ac005a8f3819\",\"endpoint\":\"https://amsktusct3.kusto.windows.net\",\"dataFactoryName\":\"MediaDataPRODServiceFabricMigration\"}' and the runtime context is '{\"servicePrincipalId\":\"71acf48e-a842-4d99-8be3-ac005a8f3819\",\"endpoint\":\"cconcat(\\\"https://amskt\\\",@item().DataCenterName,\\\".kusto.windows.net\\\")\",\"dataFactoryName\":\"MediaDataPRODServiceFabricMigration\"}'.., status code: BadRequest.", "failureType": "UserError", "target": "Copy data1" }

    The linkedService json code is,
    {
        "name": "KustoServers",
        "type": "Microsoft.DataFactory/factories/linkedservices",
        "properties": {
            "parameters": {
                "ServerName": {
                    "type": "String"
                }
            },
            "annotations": [],
            "type": "AzureDataExplorer",
            "typeProperties": {
                "endpoint": "@{linkedService().ServerName}",
                "tenant": "72f988bf-86f1-41af-91ab-2d7cd011db47"
    …"
            }
        }
    }

    And from the error message, the value can be retrieved at runtime. In this case it is https://amsktusct3.kusto.windows.net.

    Someone can help me to address the issue?

    Thanks,
    Steven

    Friday, September 27, 2019 9:50 PM

All replies

  • Hi

    As per the above code, few elements seems to be missing. Can you look in to the below code try adding the missing elements and let me know how it goes.

    {
        "name": "AzureDataExplorerLinkedService",
        "properties": {
            "type": "AzureDataExplorer",
            "typeProperties": {
                "endpoint": "https://<clusterName>.<regionName>.kusto.windows.net ",
                "database": "<database name>",
                "tenant": "<tenant name/id e.g. microsoft.onmicrosoft.com>",
                "servicePrincipalId": "<service principal id>",
                "servicePrincipalKey": {
                    "type": "SecureString",
                    "value": "<service principal key>"
                }
            }
        }
    }

    Hope this is helpful !!

    Thank you


    If this post helps to resolve your issue, please click the "Mark as Answer" of that post and/or click Answered "Vote as helpful" button of that post. By marking a post as Answered and/or Helpful, you help others find the answer faster.

    Sunday, September 29, 2019 1:01 PM
  • Hi Steven,

    You can do the following to achieve the same (I am using a REST connection for example):

    • Create a Linked Service with some static values and save it.
    • In the Linked Services tab, click on the code icon (highlighted) of the Linked Service you just created :
    • Within properties, add an attribute "parameters" in the following form  :
    {
    "name": "RestServiceWithParameters",
    "type": "Microsoft.DataFactory/factories/linkedservices",
    "properties": {
    "type": "RestService",
    "annotations": [],
    "typeProperties": {
    "url": "https://reqres.in/api/users?page=@linkedService.pageNo",
    "enableServerCertificateValidation": false,
    "authenticationType": "Anonymous"
    },
    "parameters": {
    "pageNo": {
    "type": "String"
    }
    }
    }
    }
    • You can also reference this parameter (pageNo) now as @linkedService.pageNo as shown in the above example. 
    • Create a dataset using this connection. You would be asked to pass a value for the Linked Service (pageNo). You can parameterize the dataset too. To do so, you can pass a value to the parameter as shown below :
    • You can add this parameter to the dataset (pageNum) by clicking on the add dynamic content while assigning value to the Linked Service parameter (pageNo). You will see a '+' icon to add a dataset parameter as shown below :
    • Voila, you are done. In your copy activity or any activity, you would now have to just pass a value to the dataset parameter (pageNum) which would pass that value to the Linked Service (pageNo) as shown below :

    Hope this helps.

    Monday, September 30, 2019 9:44 AM
  • I had all settings, but I removed 
    "servicePrincipalKey": {
                    "type": "SecureString",
                    "value": "<service principal key>"
                } and put ... in the origin thread.
    Monday, September 30, 2019 11:41 PM
  • Does it work now Steven?
    Tuesday, October 1, 2019 4:17 AM
  • Hi Steven,

    Just wanted to check - was the above suggestion helpful to you? If yes, please consider upvoting and/or marking it as answer. This would help other community members reading this thread.
    Wednesday, October 9, 2019 10:23 AM
  • Hi Steven,

    We haven't heard back from you in quite some time. Was the above suggestion helpful to you? If yes, please consider upvoting and/or marking it as answer. This would help other community members reading this thread.
    Friday, October 18, 2019 8:53 AM