locked
Data Factory - OData - query with request header RRS feed

  • Question

  • We have Azure data factory copy activity

    source: Microsoft Dynamics CRM Online 2016 (OData) we use the web api web services

    sink: Azure Blob Storage

    My activity typeProperties for source as follows,

    "typeProperties": {
                        "source": {
                            "type": "RelationalSource",
                            "query": "$$Text.Format('?$select=name,accountcode)"
                        }

    Is it possible to send Request headers with OData query? I want to request something like following

    GET [Organization URI]/api/data/v8.1/accounts?$select=name,accountcode&$top=1 HTTP/1.1
    Prefer: odata.include-annotations="OData.Community.Display.V1.FormattedValue"
    
    

    So that in json response I will get something like following, so that I have the code and description.

      
    "accountcode": 1,
      "accountcode@OData.Community.Display.V1.FormattedValue": "Default Value",
    
    


    Subash.S


    • Edited by Subash.S Friday, December 16, 2016 3:11 PM formatting
    Friday, December 16, 2016 3:10 PM

Answers

  • It's currently not supported by Azure data factory.

    We will put it into improvement list. Thanks for feedback.

    • Marked as answer by Subash.S Tuesday, December 20, 2016 9:37 AM
    Monday, December 19, 2016 7:18 AM

All replies

  • It's currently not supported by Azure data factory.

    We will put it into improvement list. Thanks for feedback.

    • Marked as answer by Subash.S Tuesday, December 20, 2016 9:37 AM
    Monday, December 19, 2016 7:18 AM
  • Is this still case now (Sept 2017), is it not possible to use the query to limit the number of columns that are being retrived through OData from Dynamics 365?

    When setting up a one-off copy pipeline to load data from Dynamics to Data Lake Store it indicates that there are unsupported data types and must be filitered out using the Query section, however I have so far been unable to get a query working to retrieve the other, supported, fields.

    Thanks!

    Matt


    Thursday, September 7, 2017 10:51 AM
  • Instead of pipeline focus on the input and output datasets - add "structure":[ {}, {} ] clause in the datasets. You need to define/list the columns you need other than the unsupported data type columns.

    "structure": [      {        "name": "createdon",        "description": "DateTimeOffset"      },      {        "name": "name",        "description": "String"      }]


    Subash.S

    Thursday, September 7, 2017 11:23 AM
  • OK, thank you for the guidance, I will try that!

    Matt

    Friday, September 8, 2017 9:10 AM
  • Hi

    Is this issue resolved now? Do we have out of box functionality for adding headers in to get descriptive data in oData query?

    Friday, November 29, 2019 11:11 AM
  • This is an old thread , please open a new thread as lot of changes have come through since 2016 .

    Also can you please let us know which version of ADF you talking about v1 or v2 ?


    Thanks Himanshu

    Monday, December 2, 2019 6:29 PM
  • Is the issue resolved . We could successfully load from SAP ECC using SAP ECC connector but not able to set request header prefer:Odata.track-changes.It's very urgent .
    Friday, January 31, 2020 2:08 AM