locked
\r not permissable value of format's rowdelimeter of Dataset with type Azure Blob RRS feed

  • Question

  • I have a csv file and importing the data into SQL Azure DW. All is fine but i realized that i am getting Char(13) in the last column value so itried to change the rowdelimeter to \r\n but while deploying the dataset \r disappears automatically. While only placing the \r value while uploading it gets converted to \n.

    I can handle this in TSQL but what is an elegant way of handling it in data factory especially in blob storage dataset. I am providing the dataset json for reference as well.

    csv file(column header row has been removed intentionaly)

    "10","36","3/25/2013 2:14:05 PM","True","0.00","20.00","0.00","0.00","0.00","0.00","0.00"
    "22","71","4/24/2013 10:31:38 AM","True","0.00","0.00","0.00","20.00","0.00","0.00","0.00"

    Dataset Json

    {    "name": "BlobFeesFile",    "properties": {        "published": false,        "type": "AzureBlob",        "linkedServiceName": "LiftCreditStorage",        "typeProperties": {            "fileName": "plm_fees.csv",            "folderPath": "currentday/",            "format": {                "type": "TextFormat",                "rowDelimiter": "\n",                "columnDelimiter": ",",                "nullValue": " ",                "quoteChar": "\""            }        },        "availability": {            "frequency": "Day",            "interval": 1,            "offset": "09:00:00"        },        "external": true,        "policy": {}    }}


    Thursday, October 29, 2015 4:59 PM

Answers

  • Sorry for the very late response on this question. Refer to the document: https://azure.microsoft.com/en-us/documentation/articles/data-factory-azure-blob-connector/#specifying-textformat.

    You could remove the rowDelimiter setting and by default the "\r\n" will be treated as row delimiter. But when specifying the rowDelimiter property, only one character is allowed and only the first character is honored if multiple characters are specified.

    Thursday, March 10, 2016 8:35 AM

All replies

  • I can confirm, I have the same issue!

    The Documentation notes possible values for rowDelimiter as \r, \n, \r\n

    (https://azure.microsoft.com/en-us/documentation/articles/data-factory-azure-blob-connector/)

    Must be a bug?

    Can anybody confirm?

    Hannes

    • Edited by hmayer1980 Saturday, October 31, 2015 6:40 AM clarification
    Saturday, October 31, 2015 6:38 AM
  • I would appreciate any comment on this question…


    http://www.hmayer.net/

    Sunday, November 15, 2015 7:37 PM
  • Sorry for the very late response on this question. Refer to the document: https://azure.microsoft.com/en-us/documentation/articles/data-factory-azure-blob-connector/#specifying-textformat.

    You could remove the rowDelimiter setting and by default the "\r\n" will be treated as row delimiter. But when specifying the rowDelimiter property, only one character is allowed and only the first character is honored if multiple characters are specified.

    Thursday, March 10, 2016 8:35 AM