locked
Can we perform lookup activity in Azure Data Factory on the data stored in CSV file which is stored in Azure Blob storage? RRS feed

  • Question

  • I have a requirement where I need to store a value in CSV file in Azure Blob Storage, then read the value(as parameter) in Copy activity for filtering data from source using SQL. The CSV file will contain only a single value.

    I am trying to do it by doing lookup activity on CSV file and then use that value in Copy Activity source SQL.

    Can anyone help me in that? Is there any other way to do the same?

    Wednesday, May 29, 2019 10:06 AM

Answers

  • Hi Prachee,

    Yes, you can store values from a csv file in Blob storage. To do so, you will have to chain a Lookup activity with a set variable activity as shown below :

    In the set variable activity, you can use dynamic content to access the output of the Lookup activity to get the value you want to get. It could be something like :

    @string(activity('Lookup1').output.firstRow.size)

    Quick Tip - Try running the pipeline in debug mode to see the structure of the output of Lookup activity. Once you have the structure, you can formulate a dynamic expression as shown above, and store it in a variable.

    Note: Please use the string function to convert the object to a string (that can be stored in variables).

    Hope this helps.

    Wednesday, May 29, 2019 11:13 AM

All replies

  • Hi Prachee,

    Yes, you can store values from a csv file in Blob storage. To do so, you will have to chain a Lookup activity with a set variable activity as shown below :

    In the set variable activity, you can use dynamic content to access the output of the Lookup activity to get the value you want to get. It could be something like :

    @string(activity('Lookup1').output.firstRow.size)

    Quick Tip - Try running the pipeline in debug mode to see the structure of the output of Lookup activity. Once you have the structure, you can formulate a dynamic expression as shown above, and store it in a variable.

    Note: Please use the string function to convert the object to a string (that can be stored in variables).

    Hope this helps.

    Wednesday, May 29, 2019 11:13 AM
  • Thanks Chirag. That helped :)
    Tuesday, June 4, 2019 5:13 AM
  • Glad to be of help Prachee. Please consider marking as answer the above suggestion as it might benefit other community members reading this thread.
    Tuesday, June 4, 2019 9:39 AM