none
Copy Activity - error code 2012 "An item with the same key has already been added."

    Question

  • I'm trying to run a copy activity with the source data set being an HTTP request which returns a JSON string and sinking that into an Azure SQL DB table. Any suggestions on what might be the issue? 

    Additional Information:

    Pipeline Run ID: 8bc5f24b-b85e-4a31-aef1-2c721690bfed

    ------------------ Copy Activity JSON --------------------

    {
    "name": "LUIS to SQL",
    "properties": {
    "activities": [
    {
    "name": "CopyLuisToSQL",
    "type": "Copy",
    "policy": {
    "timeout": "7.00:00:00",
    "retry": 0,
    "retryIntervalInSeconds": 500,
    "secureOutput": false
    },
    "typeProperties": {
    "source": {
    "type": "HttpSource",
    "httpRequestTimeout": "10:00:00"
    },
    "sink": {
    "type": "SqlSink",
    "writeBatchSize": 10000
    },
    "enableStaging": true,
    "dataIntegrationUnits": 0,
    "translator": {
    "type": "TabularTranslator",
    "columnMappings": {
    "Query": "Query",
    "BookFlight": "BookFlight",
    "None": "None",
    "OrderEspressoDrink": "OrderEspressoDrink",
    "OrderFood": "OrderFood",
    "GetForecast": "GetForecast"
    }
    }
    },
    "inputs": [
    {
    "referenceName": "LUIS_GetLogs",
    "type": "DatasetReference"
    }
    ],
    "outputs": [
    {
    "referenceName": "LUIS_Logs",
    "type": "DatasetReference"
    }
    ]
    }
    ]
    },
    "type": "Microsoft.DataFactory/factories/pipelines"
    }
    Friday, June 29, 2018 12:33 AM

Answers

  • Hi, 

    In your pipeline definition, "httpRequestTimeout" is specified in both the HTTP dataset and copy activity's source. According to the following document, it should be defined in copy activity's source. Can you try remove the "httpRequestTimeout" property from dataset definition ?

    https://docs.microsoft.com/en-us/azure/data-factory/connector-http

    Friday, June 29, 2018 3:10 AM

All replies

  • Hi, 

    In your pipeline definition, "httpRequestTimeout" is specified in both the HTTP dataset and copy activity's source. According to the following document, it should be defined in copy activity's source. Can you try remove the "httpRequestTimeout" property from dataset definition ?

    https://docs.microsoft.com/en-us/azure/data-factory/connector-http

    Friday, June 29, 2018 3:10 AM
  • That was it.. thank you!
    Friday, June 29, 2018 9:53 PM