none
How use file names column in OUTPUT statement?

    Question

  • Hi!
    When I extract data from files in way like in example code, how I can use filename column in OUTPUT statement (in destination path)? Is this possible?

    @searchlog =
        EXTRACT UserId          int
              , Start           DateTime
              , Region          string
              , Query           string
              , Duration        int
              , Urls            string
              , ClickedUrls     string
              , date            DateTime
              , filename        string
        FROM "/Sample/Data/{filename:*}.txt"
        USING Extractors.Tsv();


    Wednesday, April 20, 2016 2:53 PM

Answers

  • So there is no other way to save data to many files? Only multiplied OUTPUT and EXTRACT statements?
    Correct. This feature is under development at the moment and will become available at some point later this year. Here is the customer feedback item you can add your vote to: https://feedback.azure.com/forums/327234-data-lake/suggestions/10550388-support-dynamic-output-file-names-in-adla

    Michael Rys

    Wednesday, April 20, 2016 6:22 PM
    Moderator

All replies

  • Currently, the OUTPUT statement can output to a single file only, so parameterizing the filename is not possible yet. Supporting OUTPUT to multiple files is currently tracked on our plans.

    Wednesday, April 20, 2016 3:59 PM
  • So there is no other way to save data to many files? Only multiplied OUTPUT and EXTRACT statements?
    Wednesday, April 20, 2016 4:33 PM
  • So there is no other way to save data to many files? Only multiplied OUTPUT and EXTRACT statements?
    Correct. This feature is under development at the moment and will become available at some point later this year. Here is the customer feedback item you can add your vote to: https://feedback.azure.com/forums/327234-data-lake/suggestions/10550388-support-dynamic-output-file-names-in-adla

    Michael Rys

    Wednesday, April 20, 2016 6:22 PM
    Moderator