none
Query U-SQL with Powershell RRS feed

  • Question

  • Hi,


    In our azure environment we have 99 U-SQL jobs, and we run into a frequent problem where some of the jobs are not running automatically.


    What command could I use to check for jobs that have not run at all.


     I know you can run "Get-AdlJob -Account *********  -SubmittedAfter $d -result succeeded" to check for succeeded/ failed jobs, but wan't to check for ones that have not run at all.


    Setting the result to "none" doesn't appear to work either.


    Thanks

    J

    • Moved by Femisulu Wednesday, October 10, 2018 4:31 PM better suited here
    Wednesday, October 10, 2018 1:34 PM

All replies

  • Hello,

    You can run the following cmdlet to know the last execution date and status of each one of your runbooks.

    https://docs.microsoft.com/en-us/powershell/module/azurerm.automation/get-azurermautomationjoboutputrecord?view=azurermps-6.10.0


    Hope this helps.



    Regards,

    Alberto Morillo
    SQLCoffee.com


    Thursday, October 11, 2018 4:51 AM
  • Hi Alberto,

    Thank you for replying to me, correct me if I am wrong but I want to know if a U-SQL job has not run that day. So I am not really interested if the runbook has run or not.

    In effect I want I want run a script that generates a list of the U-SQL jobs that did not run, not particularly interested if the runbook has run or not.

    Let me know if it makes sense.

    Thanks again for replying to me.

    J

    Wednesday, October 24, 2018 10:34 AM
  • Hi,

    You can get the details with command Get-AdlJob

    List jobs by job state

    Get-AdlJob -Account $adla -State Running

    Using the -State parameter. You can combine any of these values:

    Accepted
    Compiling
    Ended
    New
    Paused
    Queued
    Running
    Scheduling
    Start

    OR

    List jobs by job result

    Get-AdlJob -Account $adla -State Ended -Result Failed

    Use the -Result parameter to detect whether ended jobs completed successfully. It has these values:

    Cancelled
    Failed
    None
    Succeeded

    Here Account is the Data Lake Store account


    Cheers,

    Monday, October 29, 2018 7:59 AM
  • Hi Amit,

    Thank you for responding to me, however I managed to find a solution to this.

    I generated a list of all the U-SQL jobs I know that need to run to a CSV. I then ran a query that checks the U-SQL that has run in the last day to which I output these results to another file. I then run another query to check the differences of the files and output the differences to another CSV and what ever is in that file I know needs to be re-run.

    Let me know if you wish to see the contents of the script.

    Thanks again.

    Monday, October 29, 2018 8:09 AM