Multiple HTTP endpoint to map dynamically


  • Hello,

    I would like to get some help on my requirement. My Pipeline currently able to iterate into 3 HTTP endpoint each with different columns being returned as json file. My problem is I cannot use the JSON Path settings in my HTTP dataset due to this dynamic output, hence I am not able to convert the json file into a pipe delimited format to be inserted into SQL DW.

    Can someone help me on this. Thank you

    Thursday, October 4, 2018 12:45 AM

All replies

  • Hello,

    Do you have any code snippets to share to demonstrate what you are trying to accomplish?

    Tuesday, October 9, 2018 10:29 PM
  • Hello,

    Thank you for your reply. Here is a part of the whole pipeline. I have a foreach that takes a parameterized query (@{item().Cols) and executes a get method to an HTTP endpoint. Due to different results given by the method, I can't map directly to a SQL table and exported instead to a json file (still created a c# program to transform this file to a pipeline delimited). Is there a way that I can remove the c# program and dynamically map the json file to a SQL table?

                        "activities": [
                                "name": "HTTPEndpoint_To_Txt_File",
                                "type": "Copy",
                                "policy": {
                                    "timeout": "7.00:00:00",
                                    "retry": 0,
                                    "retryIntervalInSeconds": 30,
                                    "secureOutput": false,
                                    "secureInput": false
                                "typeProperties": {
                                    "source": {
                                        "type": "HttpSource",
                                        "httpRequestTimeout": "00:01:40"
                                    "sink": {
                                        "type": "BlobSink"
                                    "enableStaging": false,
                                    "dataIntegrationUnits": 0
                                "inputs": [
                                        "referenceName": "IN_CCA_HTTP",
                                        "type": "DatasetReference",
                                        "parameters": {
                                            "paramLocation": " ",
                                            "paramQuery": {
                                                "value": "@{item().Cols}",
                                                "type": "Expression"

    Tuesday, October 9, 2018 10:59 PM