U-SQL Vertex errors - Debug Recommendations + Feature request


  • When using U-SQL to merge many files together in a partitioned directory structure errors sometimes happen due to a bad value, schema structure, etc.  When this happens we get an error like: Vertex SV1_Extract_Partition[0][17], followed by a more descriptive error that tells you what column and what values had the error.  

    Based on this, I assume this means the error happened in the 17th file, but there's no easy way to open the 17th or download it for local debugging in Visual Studio.  I have to determine which is the 17th file in a manual way.  It would be VERY helpful if on the Data tab it highlighted or bolded the file that actually failed, and also showed the Data Lake directory path of each file in the vertex.  Right now to get the folder path you have to open the file and then click properties.  I think it would be helpful to have an icon to click next to Size that does a popup with the file path, along with highlighting the file that failed.  It would be easy to implement and save us time trying to investigate which file actually caused the failure.  If there's an easier way to already do this please let me know because I spend a lot of extra time tracking down which file actually failed! Thanks!

    • Edited by FrankMn Tuesday, July 10, 2018 9:32 PM spelling
    Monday, July 9, 2018 4:41 PM