none
Custom Activity Output values for use in sequential task

    Question

  • I have a simple custom code app that is outputting a generic string value "my custom output value".  I need to use this value in an activity the custom code activity.  Documentation around how to do this seems to be somewhat lacking.  I can see that when I write a value in my application it gets stored in the ADF job output file "stdout.txt".  For instance, if I go into Blob storage and find the stdout.txt file for the job that was executed I see a value of "my custom output value" in that file.

    How should I be accessing this value from within the next ADF activity? 

    Can I set a Pipeline Variable and then assign the custom code value to the variable?  Or can I directly access the stdout.txt value by accessing the custom code output properties?  I'm a bit lost here on how to do this correctly. Thanks!

    • Edited by FrankMn Monday, November 19, 2018 4:39 PM added details
    Monday, November 19, 2018 4:38 PM

Answers

  • You can find the output structure in output panel.

    You can get the output file url by @activity('customActivityName').output.outputs[0]

    But I suggest you to use activityRunId to access the file as the output file path follow a certain pattern: https://<container>.blob.core.windows.net/adfjobs/<activityRunId>/output/stdout.txt

    You can set a dataset with a parameter called runId.

    Set the file path as below:

    Then in your following activity such as lookup or getmetadata, reference that dataset, and set the runId value as @activity('CustomActivityName').ActivityRunId.

    That way you can read the blob file.

    • Marked as answer by FrankMn Tuesday, November 20, 2018 2:16 PM
    Tuesday, November 20, 2018 2:25 AM

All replies

  • Hi,

    Method1

    Write the output as key/value pair in the outputs.json and you can access it in ADF as follows, @activity('customactivityname').output.customOutput.key Method2

    Try to write the value in a new file and upload the file into blob-storage in the custom code. If more values, then use some structured format while writing the output that makes the next step easier.

    The next step is to use lookup activity to read the blob file and use the values as needed.

    If you want to set the values to a variable, use set-variable activity after the lookup activity.


    Regards,

    Shahila






    • Edited by Shahila Monday, November 19, 2018 10:03 PM
    Monday, November 19, 2018 9:42 PM
  • I found a <g class="gr_ gr_40 gr-alert gr_spell gr_inline_cards gr_run_anim ContextualSpelling ins-del multiReplace" data-gr-id="40" id="40">simiar</g> thread. Check this
    Monday, November 19, 2018 10:09 PM
  • Hi,

    Method1

    Write the output as key/value pair in the outputs.json and you can access it in ADF as follows, @activity('customactivityname').output.customOutput.key Method2

    Try to write the value in a new file and upload the file into blob-storage in the custom code. If more values, then use some structured format while writing the output that makes the next step easier.

    The next step is to use lookup activity to read the blob file and use the values as needed.

    If you want to set the values to a variable, use set-variable activity after the lookup activity.


    Regards,

    Shahila






    What is output.customOutput because ADF is saying it doesn't exist.  What does customOutput represent? 
    Monday, November 19, 2018 10:43 PM
  • You can find the output structure in output panel.

    You can get the output file url by @activity('customActivityName').output.outputs[0]

    But I suggest you to use activityRunId to access the file as the output file path follow a certain pattern: https://<container>.blob.core.windows.net/adfjobs/<activityRunId>/output/stdout.txt

    You can set a dataset with a parameter called runId.

    Set the file path as below:

    Then in your following activity such as lookup or getmetadata, reference that dataset, and set the runId value as @activity('CustomActivityName').ActivityRunId.

    That way you can read the blob file.

    • Marked as answer by FrankMn Tuesday, November 20, 2018 2:16 PM
    Tuesday, November 20, 2018 2:25 AM
  • You can find the output structure in output panel.

    You can get the output file url by @activity('customActivityName').output.outputs[0]

    But I suggest you to use activityRunId to access the file as the output file path follow a certain pattern: https://<container>.blob.core.windows.net/adfjobs/<activityRunId>/output/stdout.txt

    You can set a dataset with a parameter called runId.

    Set the file path as below:

    Then in your following activity such as lookup or getmetadata, reference that dataset, and set the runId value as @activity('CustomActivityName').ActivityRunId.

    That way you can read the blob file.

    This was helpful, thank you.  I was able to create the dataset and get the output values from the custom code by following this example.  Thank you!
    Tuesday, November 20, 2018 2:16 PM
  • Bo Xiao's comment was the best I could find anywhere (documentation or online) for attempting to pass an activity's runid to a following activity. Though his solution with setting a parameter etc did not work for me, setting the first activity's output dataset's File path to this worked for me:

    @CONCAT('adfjobs/',activity('FirstActivityName').ActivityRunId,'/output')

    Since the ActivityId changes from run to run and the file path was otherwise static, I needed to pass the specific ActivityRunId into the File Path.

    Tuesday, February 12, 2019 7:14 PM