none
U-SQL Runtime Exception causing Runaway job in cloud

    Question

  • The following script has a error, but the runtime never catches it and the job never fails but keeps running forever:

    @employees = 
        SELECT * FROM ( VALUES
            (1, "Noah",   "Engineering", 100, 10000),
            (2, "Sophia", "Engineering", 100, 20000),
            (3, "Liam",   "Engineering", 100, 30000),
            (4, "Emma",   "HR",          200, 10000),
            (5, "Jacob",  "HR",          200, 10000),
            (6, "Olivia", "HR",          200, 10000),
            (7, "Mason",  "Executive",   300, 50000),
            (8, "Ava",    "Marketing",   400, 15000),
            (9, "Ethan",  "Marketing",   400, 10000) )
        AS T(EmpID, EmpName, DeptName, DeptID, Salary);

     
    @result=
        SELECT
            EmpName, 
            DeptName.Substring(4) AS Dept4,
            SUM(Salary) OVER( PARTITION BY DeptName ) AS SalaryByDept,
            SUM(Salary) OVER( ) AS SalaryAllDepts
        FROM @employees;


    OUTPUT @result TO "/Samples/wfresult2.csv" 
        USING Outputters.Csv();



    • Edited by SanjeevD Wednesday, April 27, 2016 2:41 PM
    Wednesday, April 27, 2016 12:06 PM

Answers

  • Hi Sanjeev

    Thanks for reporting the issue (and my apologies for the late reply, but the forums didn't let us reply).

    Ihave repro'ed it and are now investigating. In my case the job failed with a runtime error that had no useful error information after about 10 minutes.

    Thanks again and please let me know if this issue is blocking you.


    Michael Rys


    Friday, April 29, 2016 11:02 PM
    Moderator

All replies

  • Hi Sanjeev

    Thanks for reporting the issue (and my apologies for the late reply, but the forums didn't let us reply).

    Ihave repro'ed it and are now investigating. In my case the job failed with a runtime error that had no useful error information after about 10 minutes.

    Thanks again and please let me know if this issue is blocking you.


    Michael Rys


    Friday, April 29, 2016 11:02 PM
    Moderator
  • After further investigation, you are causing the error with the following C# expression:

    DeptName.Substring(4)

    Since some of the DeptName values (like HR) are shorter, C# raises an error. Depending on what you want as a result for these cells, you can replace the above expression with

    DeptName.Substring(Math.Min(4,DeptName.Length))

    to produce a zero-length string,

    or with

    DeptName.Length<4? null: DeptName.Substring(4)

    to produce null.


    Michael Rys

    Saturday, April 30, 2016 12:14 AM
    Moderator