U-SQL Outputters.text() failing in production servers with no inner error message


  • The sample project "AvroExamples" from Microsoft's U-SQL repo:

    fails when run on production servers. The failure is a very generic "The vertex failed too many times", as it doesn't provide any inner error that would hint what is going on.

    The same U-SQL query, with the same input, works fine locally. 

    After a few trials and errors, the culprit of the error is the use of backward slashes in this U-SQL line:

         OUTPUT @cnt TO @output_file USING Outputters.Text();

    where output_file is  DECLARE @output_file string = @"\output\twitter.csv";

    Replacing the second backslash with a forward slash :
     OUTPUT @cnt TO @"\output/twitter.csv" USING Outputters.Text();

    fixes the problem.

    So somehow, Outputters gets confused by the use of backward slashes. 

    How to reproduce:

      - Clone the git repo.

      - Follow the setup instructions of the AvroExamples project : Create the database, register the assemblies, copy the input sample file to Datalake. 

      - Then simply execute the "3-SimpleAvro" query on a ADLA, from Visual Studio 2017,

      - The query will fail at "executing phase", with "User: VertexFailedTooMany", Description "Vertex SV1_extract[0] failed 6 times". Message: "Vertex failed too many times".

    Friday, March 23, 2018 6:24 AM