none
Minimal information required in the Rest API header body to reflect changes to the data source RRS feed

  • Question

  • Can Data Catalog Rest API reflects the changes of a registered source (SQL data warehouse for instance), without specifying all the changes (eg added column, added rows in preview) in the header body? 
    When I re-register a source using the Registration tool, any changes to the source is automatically reflected. However, with the Rest API, it seems that I have to specify all my changes to the schema in the header body, as supposed to just adding in the basic information such as server and database.
    Tuesday, September 17, 2019 4:27 AM

All replies

  • Hi there,

    Unfortunately, the recommended way to reflect changes in the data source is to re-register the data source as mentioned in the FAQ here :

    https://docs.microsoft.com/en-us/azure/data-catalog/data-catalog-frequently-asked-questions#how-do-i-update-the-registration-for-a-data-asset-so-that-changes-in-the-data-source-are-reflected-in-the-catalog

    How do I update the registration for a data asset so that changes in the data source are reflected in the catalog?

    To update the metadata for data assets that are already registered in the catalog, simply re-register the data source that contains the assets. Any changes in the data source, such as columns being added or removed from tables or views, are updated in the catalog, but any annotations provided by users are retained.

    I would recommend you to provide feedback at the feedback forum. All the feedback you share, is closely monitored by the Data Catalog Product team and implemented in future releases.

    Also, Regarding service general availability, I would suggest to keep an eye on Azure updates.
    Azure updates provide information about important Azure product updates, roadmap, and announcements. 

    Hope this helps.

    Tuesday, September 17, 2019 7:06 AM
  • Hello, thank you for your prompt reply.

    We can re-register via the Data Catalog GUI, but we are seeing there is an "automated" way to automatically refresh the Data Catalog on a regular basis automatically. Hence that is why we are looking at using the RESTful API's in a program that can be regularly invoked to refresh the data.

    Any suggested designs are much appreciated.

    Cheers,
    Peter

    Tuesday, September 17, 2019 9:46 AM
  • Hi Peter,

    You can try doing the same using REST APIs as follows :

    Register or Update

    Registers a new data asset or updates an existing one if an asset with the same identity already exists. The items can optionally contain ETag values to enable optimistic concurrency control for them.

    Get started sample on GitHub

    Request

    <button aria-label="Copy code" class="action" data-bi-name="copy" style="box-sizing:inherit;font-family:inherit;font-size:12.8px;margin:0px;cursor:pointer;padding:2px 10px;border-left-width:1px;display:flex;align-items:center;">Copy</button>
    POST https://api.azuredatacatalog.com/catalogs/{catalog_name}/views/{view_name}?api-version={api-version}  
    

     Note

    Some HTTP client implementations may automatically re-issue requests in response to a 302 from the server, but typically strip Authorization headers from the request. Since the Authorization header is required to make requests to ADC, you must ensure the Authorization header is still provided when re-issuing a request to a redirect location specified by ADC. Below is sample code demonstrating this using the .NET HttpWebRequest object.

    Uri parameters

    Name Description Data Type
    catalog_name Name of the catalog, or "DefaultCatalog" to use the default catalog. String
    view_name Name of Data Asset View. String
    api-version The API version. String

    POST example

    POST https://api.azuredatacatalog.com/catalogs/DefaultCatalog/views/tables?api-version=2016-03-30

    Header

    Content-Type: application/json
    x-ms-client-request-id: 13c45c14…46ab469473f0
    Authorization: Bearer eyJ0eX ... FWSXfwtQ

    Reference - https://docs.microsoft.com/en-us/rest/api/datacatalog/data-catalog-data-asset#register-or-update

    Hope this helps.

    Wednesday, September 18, 2019 8:37 AM
  • Hi there,

    Just wanted to check - was the above suggestion helpful to you? If yes, please consider upvoting and/or marking it as answer. This would help other community members reading this thread.
    Monday, September 23, 2019 8:29 AM
  • Hi there,

    We haven't heard from you in quite some time. Was your issue resolved?

    Thursday, October 3, 2019 11:19 AM