Source Table Schema Change - No Error, Data Copied into Wrong Columns RRS feed

  • Question

  • I've got a dataset pointing to a particular table in an on-premise MySQL database.

    Data is moved from this table to a source table in an Azure SQL database nightly.

    Recently, the schema in the source MySQL table changed (a column was added), yet ADF continued to "work" without throwing an error, copying the data into incorrect columns in the destination table for all columns after the added column, shifted one column over alphabetically.

    I feel that this behavior is quite odd. Personally, I'd expect the schema change to be seen and an error thrown, cancelling the data slice. It wrote the data from the new column in a column with an entirely different name, and the data in the last column was lost due to not having a column in the destination to write to. There are full mappings defined in the pipeline, so it seems that the mappings were simply ignored.

    Is there a strict setting that I'm missing somewhere for this Copy activity? I'd like ADF to freak out and let me know if something changes, rather than just throw the data wherever it thinks it'll fit.

    Tuesday, January 24, 2017 4:04 PM