locked
ADF Copy doesn't like linefeeds inside the data. RRS feed

  • Question

  • Executive Summary:

    ADF don't deal well with CSVs with linefeeds inside data.  Please to fix.

    Details:

    I recently attempted to create an ADF which took a rather large CSV and loaded it into a Azure SQL table.

    After the usual debugging it foundered on the fact that within one of the CSV's columns there is "rich text" i.e. \r\n are acceptable input elements.  Needless to say changing the "rowDelimiter" is moot as this is a CSV file. I do find it mildly confounding that Excel itself doesn't get confused but ADF does, but ADF is a very young technology — as opposed to Excel.

    At any rate: "Data Copy Wizard" chokes on the file's size (or that same \n\r issue), and I am unable to export from the source as JSON but could do XML; but, ADF doesn't do XML yet.  In sum this means I'll need to HQL this problem, which is a mild disappointment as I had hoped to be able to "configure n' go".


    Thursday, April 28, 2016 1:19 PM

All replies

  • Thanks for the question. In fact, ADF Copy could handle the linefeeds inside the data if the data is quoted. You could specify the quote character property so that the quoted linefeeds would be treated as data value instead of row delimiter.
    Tuesday, May 10, 2016 2:00 AM