Using wildcards to find files using the API


  • So I can use `_adlsFileSystemClient.FileSystem.ListFileStatus("ACCOUNTNAME", "/somefolder");` to get files in a specific directory, but is it possible to use wildcards as well.

    Specifically, I am looking to get files that match "/main/{id}/subfolder/file.txt".

    Right now, the best I can tell, is that I would need to make API call `ListFileStatus("/main")`, then make subsequent calls for each subdirectory `ListFileStatus("/main/{id}")` and then another round of API calls to `GetFileStatus("/main/{id}/subfolder/file.txt") `, and then one more round to read in all the files.

    Based on my experience with Data Lake API call times, this is going to be an expensive (in time) operation. Though I would hate to make this many calls with the fastest API. Even a recursive option on `ListFileStatus` would be helpful.

    Ideas on how this can be improved?

    Tuesday, December 20, 2016 1:18 PM