locked
Setting error count while loading to SQL Sink RRS feed

  • Question

  • I am loading data from Azure blob storage to Azure SQL Data warehouse using ADF. During the load, I do not want the pipeline to fail until the error count is 10. Is there a way to set this ? 

    Thanks

    Thursday, December 17, 2015 5:24 AM

All replies

  • Hi Nikil,

    Currently ADF Copy doesn't support skipping error rows. Could you describe more about your scenarios so that we could prioritize thie feature?

    In the meantime, what errors you want to skip and is there any workaround before the skip feature is available?

    Monday, December 21, 2015 2:33 AM
  • I am using Copy activity to copy data from CSV file to Azure SQL DW. Some of the data types in the CSV file do not match the expected data types in SQL DW. I want to skip the error rows and continue copying data.

    Workaround right now is to copy all the data to a stage file with varchar data type and write a procedure to convert data types as needed.

    Thanks

    Monday, December 21, 2015 7:38 PM
  • Thanks for providing the detailed information. It is definitely a valid requirement, and we will further triage when to support it.
    Tuesday, December 22, 2015 2:23 AM