none
ADF V2 - Azure SQL DB Linked Service is not publishing with Git Integration RRS feed

  • Question

  • I'm having an issue with getting Azure SQL DB Linked Services to publish to the the data factory.  

    I have tried using a Linked Service that is getting credentials from KeyVault, as well as manually entering the information. In both cases it connects and the pipelines can run fine when i'm on the collaboration branch.

    I thought it might be an issue with parameters on the Linked Service, but this also does not seem to work. 

    There are 3 other Link Services. 2 KeyVaults and a Datalake Gen2, these all test fine and they are able to publish without issue. 

    so far I have: 

    1. Tested two other SQL Db Linked Services, none of them published. 
    2. Deleted the repository and recreate it. 
    3. Created the JSON manually and pushed, will connect and run the pipeline fine, but publish does not pick up the linked service 
    4. Created the Linked Service in the Web UI, will connect and run the pipeline fine, but publish does not pick up the linked service  
    5. Attempted the same in a second, test data factory and got the same results. 

    At this rate I am out of ideas, are database linked services not able to be used in the GIT integration?

    Monday, July 1, 2019 12:53 PM

All replies

  • Hello Mitchell Hamann, and thank you for your inquiry.  From your description of the issue and the exhaustive steps you have taken, it is clear that a serious investigation is needed.  Before going on to that, I want to make sure there are no misunderstandings.

    You mentioned manually creating and pushing JSON.

    When I created JSON for a dataset and then pushed it with Powershell, the dataset was published directly, bypassing the Git Integration.  The dataset was visible when I swapped the UI from GitHub view to Data Factory view.  Since the dataset was written to the published version instead of the Git repo, publishing from the Git repo would not see it.  Is this scenario a possibility?

    Tuesday, July 2, 2019 10:42 PM
    Moderator
  • hey MartinJaffer-MSFT, 

    Thank you for your response. I actually generated the Linked Service json via powershell programmatically. I have not tried deploying it out from powershell directly yet. I can try that, however it will cause an issue with CI/CD if we have to add manual steps into the process to deploy changes. Luckily it should be fairly simple. 

    I will try this out and let you know what happens. I am curious about the bug that would be causing it. It would be very helpful to have some more insight and tooling around the publishing. 

    Wednesday, July 3, 2019 5:51 PM
  • Hello Mitchell.  I finally got some clarification.  The end result of publishing, and the end result of creating a resource programmaticaly, are the same.  With that in mind, I am a little confused when you said:

    I actually generated the Linked Service json via powershell programmatically. I have not tried deploying it out from powershell directly yet.

    By this do you mean, you created the JSON definition for the Linked Service, and then pushed it to Git, and then later published from the Git, making the change to the Data Factory?  If that is the case, and that is where the sql isn't being created, then I'll escalate your issue.

    Tuesday, July 9, 2019 2:20 AM
    Moderator
  • By this do you mean, you created the JSON definition for the Linked Service, and then pushed it to Git, and then later published from the Git, making the change to the Data Factory?  If that is the case, and that is where the sql isn't being created, then I'll escalate your issue.

    Correct. I created the JSON definition using a powershell module I wrote that programmatically creates the linked Service, I then commit the JSON to the git repo and push it up. Once this is done I test it out ( connects fine ) and then publish it. For all intents and purposes the final object in the git repo is totally valid and can even be used in the pipeline. However when I go to publish it, it isn't listed. 

    Tuesday, July 9, 2019 1:06 PM
  • Is it possible that the object has already been published?
    Monday, July 15, 2019 11:33 PM
    Moderator
  • I thought that might be the case, but I ended up deleting the object, as well as creating completely new objects with different names to make sure. None of them are picked up by the publish. 
    Tuesday, July 16, 2019 12:04 PM
  • Hi Mitchell Hamann.  Since I have not been able to help you, perhaps you can try a support ticket.

    If you have a support plan you may file a support ticket, else could you please send an email to AzCommunity@Microsoft.com with the below details, so that we can create a one-time-free support ticket for you to work closely on this matter. 

    Thread URL: <MSDN Thread>Subscription ID:  <your subscription id>

    If you do have a support plan, please share the resolution here so everyone may benefit.
    Thursday, July 18, 2019 6:46 PM
    Moderator
  • Hello @Mitchell Hamann.  Are you still there?
    Friday, July 26, 2019 9:05 PM
    Moderator