locked
Call XML Data from API calls. RRS feed

  • Question

  • Hello

    I need to call XML Data from URL and store it in Azure storage and then finally to Azure SQL DB. ADF doesnt have options for copying XML Data and Im looking for other ways to do it. If you can provide some useful suggestions it would be of great help.

    Thanks


    Arun

    Friday, September 27, 2019 10:05 AM

Answers

All replies

  • Hi Arun,

    Unfortunately, as of today, the REST connector for copy activity supports only JSON response. 

    However, you can use a custom activity or an azure function activity to do it using ADF. In the custom activity/azure function, you can write code to read from the XML based API and write to Azure Storage using it's SDK.

    Hope this helps.

    Friday, September 27, 2019 11:40 AM
  • Thanks Chirag. Can this be done using LogicApps as well. And if yes which will be the best solution to implement. Either using Custom/Azure function or using LogicApps.

    And once the data is loaded into Azure Storage i can parse the XML and store it in Azure SQL DB.  Thanks.



    Arun

    Friday, September 27, 2019 1:38 PM
  • Can Azure Databricks be used as an option? Do we have any limitation on processing XML Data using Custom Activity or Azure Function activity?

    Arun

    Monday, September 30, 2019 3:59 AM
  • Hi Arun,

    Yes you can do it using Logic Apps as well. You would then use a web activity to trigger the http logic app.

    It totally depends on your use case. Azure functions and custom activities are in-built into ADF, hence offer more flexibility in terms of response returning, etc.

    Monday, September 30, 2019 6:49 AM
  • Yes you can also run an Azure Databricks notebook from ADF. THere are no limitations while using a custom activitiy or Azure function activity for fetching data from an XML based web service.
    Monday, September 30, 2019 6:51 AM
  • Thanks Chirag..So from Azure Databricks notebook activity I can write Python/Scala code to read the XML data from REST API and parse it and store in any Azure Storage..did you have any links where it has been done using Databricks..

    Arun

    Monday, September 30, 2019 7:02 AM
  • Hi Arun,

    There are a lot of resources available that you can search and find. I found one similar article using which, with a little modification, you can achieve your use case :

    https://kb.databricks.com/data/blob-data-in-xml.html#solution

    Hope this helps.

    Wednesday, October 9, 2019 10:23 AM
  • Hi Arun,

    Just wanted to check - Was the above suggestion helpful to you? If yes, please consider upvoting and/or marking it as answer. This would help other community members reading this thread.
    Friday, October 18, 2019 8:54 AM
  • Hi Arun,

    If your query is addressed, request you to mark the answer that helped you as “Mark as Answer” and "Up-Vote" so that other forum members can benefit from it. If you need any further help do let us know.

    Wednesday, October 23, 2019 9:15 AM