none
U-SQL job stuck on Combine vertex.

    Question

  • I have a U-SQL script/job that is running against millions of invoice records that runs fine until it hits a Combine vertex and then completely stops.  The script itself is a little complex, but I'm very confused why it won't run.  It runs just fine in SQL server for instance.

    I am basically joining invoices to itself to get the next invoice date for a given employee for a different market.  So I need to get the next invoice for the employee but the invoice needs to be from a different market.

    What is causing this to stop running? Is it a poorly written query? Can I accomplish it a better way?  Is it because I'm joining millions of records against itself? But then why doesn't it run when I throw more AUs at it?  I'm stuck and very confused.

    Query:

    SELECT 
    i.[EmployeeID]
    , i.[Market]
    , i.[invoicepk]
    , i.[invoicedate]
    , MIN(OtherMarkets.[invoicedate]) AS NextInvoice
    FROM @GlobalEmployeesWithInvoices AS i
    INNER JOIN
    (
    SELECT 
            [invoicedate]
            , [EmployeeID]
            , [Market]
    FROM @GlobalEmployeesWithInvoices AS i
    ) AS OtherMarkets
    ON i.[EmployeeID] == OtherMarkets.[EmployeeID]
    WHERE i.[Market] != OtherMarkets.[Market]
        AND 
          i.[invoicedate] <= OtherMarkets.[invoicedate]
    GROUP BY 
        i.[EmployeeID]
        , i.[Market]
        , i.[invoicepk]
        , i.[invoicedate];


    • Edited by FrankMn Monday, August 20, 2018 10:28 PM
    Monday, August 20, 2018 10:27 PM

All replies

  • Hi Frank,

        I reached out to product team and they think this issue should be processed via support ticket. If you have a support plan, please open up a ticket or please send us an email to azcommunity@microsoft.com with following details to hook you up with free support:
    - Subscription ID :
    - URL to this thread :


    Friday, August 24, 2018 8:29 PM
    Moderator