Copy/Decompress and ignore failures RRS feed

  • Question

  • I want to write a simple pipeline that copies gzip files in to ADLS. the pipeline fails when it encounters a corrupt file. all I want to do it skip over these files (copy compressed) and continue the process.

    Any help or advice appreciated

    Thanks in advance

    Monday, June 25, 2018 10:38 AM

All replies

  • You could use get meta data activity  + foreach activity + copy activity.

    Use get metadata activity to get all the file names which you wanted to copy, pass it to foreach. 

    Foreach will iterate the result array of get metadata activity. Inside the foreach, it is a copy activity.

    Finally, the pipeline will fail, but some of your activity in your foreach loop will success if it doesn't hit the issue you mentioned.

    Monday, June 25, 2018 1:48 PM