none
hdfs dfs -stat behavior when path not found

    Question

  • I am using some package that uses hadoop dfs -stat to check for the existence of a file. For example, it might do the following:

    hadoop dfs -stat adl://example.azuredatalakestore.net/data/fake

    In the case that this path doesn't exist, other platforms I've used will return something along the lines of 'stat: cannot stat path. No such file or directory.' However, when using HDInsight backed with Azure Data Lake I get a 404, breaking the existence check. Is this intended behavior?

    Wednesday, August 16, 2017 4:45 PM

All replies

  • Hi Andrew,

    Which other platforms did you try this out with? Also 404 is the HTTP code for not found and it seems somewhat logical that the REST API returns it. Somewhere along the stack clearly this is not being translated. I'm trying to reason about to what degree it breaks the behavior and how it affects your script.

    Thanks,

    Amit

    Thursday, August 17, 2017 1:29 AM