none
Error processing folder: input unavailable

    Question

  • Hi,

    I'm processing a folder on Azure Datalake Analytics and it throws an error which says a file in the folder is unavailable. What does this means? I'm for sure not working on the directory - I renamed the folder before processing it. And I'm processing a folder, so he is able to see the folder contains a file with that certain name, but it is 'unavailable'. Is it because I probably renamed the folder while I was writing to a file in the folder? How can I solve this issue?

    ERRORJM_PrimaryInputsUnavailable. Input 85 (adl://MyLongPath/EXPORT91(9).MIGv3-03 - Copy.csv[0]) is unavailable. Could not read. 

    FYI: It worked with the next folder I renamed.

    Kind regards,

    Filip



    • Edited by Filip VR Tuesday, April 19, 2016 3:30 PM
    Tuesday, April 19, 2016 3:26 PM

Answers

  • I did not renamed while I was processing the folder, I did it before. But I found the error. It simply was because of one of the files had a space in it's name, and that does not seems to be allowed.

    Regards,

    Filip

    • Marked as answer by Filip VR Thursday, April 21, 2016 2:57 PM
    Thursday, April 21, 2016 2:57 PM

All replies

  • I assume that you renamed the directory while the job was starting or running.

    There is a short period between the compiler checking for the existence of the files and the time the job manager that creates the actual execution graph and understands the scheduling of the work items in the job picks the physical extents of the file to provide parallel processing on files.

    If your rename happened after the compilation phase but before the JM (job manager) looks up the extents for the file, then the job manager will not find the renamed file and raise the above error.

    Note that ADLS is not a transacted file system. U-SQL will provide multi-valued concurrency control on ADLS files during the execution of the job graph (after the extents have been identified), but if changes happen in that window before, you will get errors.


    Michael Rys

    Thursday, April 21, 2016 8:23 AM
    Moderator
  • I did not renamed while I was processing the folder, I did it before. But I found the error. It simply was because of one of the files had a space in it's name, and that does not seems to be allowed.

    Regards,

    Filip

    • Marked as answer by Filip VR Thursday, April 21, 2016 2:57 PM
    Thursday, April 21, 2016 2:57 PM
  • Spaces are allowed, but since the path is a URI, you have to use URI encoding in your location string.

    We do have a bug at the moment with spaces (and other characters that need to be encoded in a URI) if you specify a file set. That is currently being fixed.


    Michael Rys

    Friday, April 22, 2016 5:21 PM
    Moderator