none
need to fetch the file size in Azure Data Factory pipeline once the data is getting loaded into ADLS RRS feed

  • Question

  • Hi All,

    I am working on Azure Databricks through which data is getting stored into ADLS and the data is in .csv file.

    I am using Azure Data Factory pipeline to achieve this.

    My requirement is that i need to fetch the file size of the file that is stored into ADLS and i have implemented the Mail task in ADF, in the same mail i need to send file size as welll.

    Kindly provide your necessary help, also let me know if it is possible

    Thanks in advance

    Pankaj


    Pankaj

    Tuesday, November 12, 2019 6:48 AM

Answers

  • You can try to use Get metadata activity where it should point to the folder/file and will return the size property of the file. 

    Use the output of above activity further in subsequent activities.


    Regards,
    Vaibhav

    Tuesday, November 12, 2019 7:04 AM
  • i am  able to read the file size, now issue that i am facing is how to pass that in parameter and send it in mail to client

    Which activity is sending emails and how? 

    You can pass the output from Get Metadata to subsequent activities by using -
    @activity(‘Get Metadata1’).output.size 


    If the response helped, do "Mark as answer" or Upvote it
    - Vaibhav

    • Marked as answer by Pankaj Mahra Monday, November 18, 2019 11:17 AM
    Friday, November 15, 2019 11:18 AM

All replies

  • You can try to use Get metadata activity where it should point to the folder/file and will return the size property of the file. 

    Use the output of above activity further in subsequent activities.


    Regards,
    Vaibhav

    Tuesday, November 12, 2019 7:04 AM
  • Hi,

    Thanks for your answer

    i am  able to read the file size, now issue that i am facing is how to pass that in parameter and send it in mail to client


    Pankaj

    Friday, November 15, 2019 10:15 AM
  • i am  able to read the file size, now issue that i am facing is how to pass that in parameter and send it in mail to client

    Which activity is sending emails and how? 

    You can pass the output from Get Metadata to subsequent activities by using -
    @activity(‘Get Metadata1’).output.size 


    If the response helped, do "Mark as answer" or Upvote it
    - Vaibhav

    • Marked as answer by Pankaj Mahra Monday, November 18, 2019 11:17 AM
    Friday, November 15, 2019 11:18 AM
  • Thanks a lot Vaibhav,

    I am using logic apps for mail task.

    and now able to execute end to end process, successfully sending the file size in mail.


    Pankaj

    Monday, November 18, 2019 11:20 AM
  • Hi @Vaibhav,

    I have one more requirement where we need to get the execution time of pipleine i.e. the time taken by a pipeline to run.

    Can you please provide any workaround for this.

    Thanks,

    Pankaj


    Pankaj

    Monday, November 25, 2019 1:40 PM