none
Azure Data Lake U-SQL job failing with error: JM_FinalMetadataOperationUserError

    Question

  • I've been running U-SQL jobs on Azure Data Lake that update partitioned tables and they run successfully, but fail at the "Finalizing" phase with the error "JM_FinalMetadataOperationUserError", the stack trace is as follows:

    Object ec2d8ab3-6df5-48eb-9169-daec48bf27a6.DeployedLogs.dbo.ConcurrentRecord__1__System.DateTime.FromBinary
    (5246322962427387904 /* 2012-02-01T12:00:00.0000000Z */) 
    already exists at Scope.MetadataService.Client.WcfBasedMetadataClient.ExecuteMetadataJob(MetadataJob mdJob) 
    at MetaDataExecutor.RealMetaDataExecutor.Execute(MetadataJob job) 
    at MetaDataExecutor.Committer.Commit(MetadataJob job)

    The error message states:

    Failed to write job meta-data due to user error
    

    What can cause this problem? Is it due to the values being added/truncated from the table?

    Would appreciate the help.


    Wednesday, June 1, 2016 11:29 AM

All replies

  • The investigation identified the root cause as a race condition between two scripts that were trying to create the same partition. See http://stackoverflow.com/questions/37560454/azure-data-lake-u-sql-job-failing-with-error-jm-finalmetadataoperationusererror for details.

    Michael Rys

    Thursday, June 16, 2016 12:11 AM
    Moderator
  • Hi, Im having the same issue here. But it's not rare. I created a powerShell script that simultaniously kicks off 31 jobs (each day of the month) 30 jobs fail and 1 succeed. But each jobs takes about an hour and then it fails. If I resubmit the same job one by one it succeeds succesfully

    It looks like you cannot run a job on the same storage simultaniously.  The source folder and the destination folder are all  different.

    Message

    Failed to write job meta-data due to user error

    Description

    Version of object xxxxxxxx-xxxx-xxxx-xxxx-95b21312acf0.master.Avro doesn't exist at Scope.MetadataService.Client.WcfBasedMetadataClient.ExecuteMetadataJob(MetadataJob mdJob) at MetaDataExecutor.RealMetaDataExecutor.Execute(MetadataJob job) at MetaDataExecutor.Committer.Commit(MetadataJob job)

    Edit: Despite the metadata error it seems like the jobs are all completed, because the output dataset is present..


    • Edited by Rik Opdam Thursday, September 20, 2018 12:48 PM
    Thursday, September 20, 2018 12:34 PM