none
Logic app to get datafactory output. RRS feed

  • Question

  • Suppose I have the following pipeline 

    and the activity "WebActivity1" is going to capture the error output of activity "validation2" and send email 

    {
       "DataFactoryName": "@{pipeline().DataFactory}",
       "EmailTo": "email@domain.com",
       "PipelineName": "@{pipeline().Pipeline}",
       "ActivityEndtime": "@{string(pipeline().TriggerTime)}",
       "ActivityErrorOutput": "@{string(activity('Validation2').output)}"            <<<< but seems here has the problem, as I got the error

    Error
    {
        "errorCode": "2108",
        "message": "{\"error\":{\"code\":\"InvalidRequestContent\",\"message\":\"The request content is not valid and could not be deserialized: 'After parsing a value an unexpected character was encountered: e. Path 'ActivityErrorOutput', line 6, position 29.'.\"}}",
        "failureType": "UserError",
        "target": "WebActivity1"
    }

    The json of the logic app is like the following :

    {
        "properties": {
            "DataFactoryName": {
                "type": "string"
            },
            "EmailTo": {
                "type": "string"
            },
            "PipelineName": {
                "type": "string"
            },
            "ActivityEndtime": {
                "type": "string"
            },
            "ActivityErrorOutput": {
                "type": "string"
            }
        },
        "type": "object"

    }

    Is that the type problem of ActivityErrorOutput???

    Monday, June 3, 2019 4:27 AM

All replies

  • Hello sakurai_db and thank you for your inquiry.  Here is what I would like you to try:
    Place a Set Variable activity between the Validation activity and the Web activity.
    Assign the output of the Validation activity to the variable. (
    "@{string(activity('Validation2').output)}")
    In the Web activity reference the new variable instead of the Validation activity's output directly.

    Please let me know if this helps.

    Monday, June 3, 2019 8:18 PM
    Moderator
  • There has been a similar thread about the body of web activity in particular.  The problem in that thread was caused by the conversion of " to \" .
    Tuesday, June 4, 2019 6:16 PM
    Moderator
  • actually , apart from validation activity , the other activity may have different kind of output/error

    Wednesday, June 5, 2019 8:51 AM
  • That is more than possible.  Only recently was a bug patched regarding the Validation activity.  The bug made it impossible to get the output of the Validation activity.

    Additionally, the output/error can change depending upon the settings of an activity.  For example, in a Lookup Activity, selecting the 'get only first row' option radically changes the output.  Take the case of querying a table returning zero rows.  When 'get only first row' is true, the activity will throw an error.  When 'get only first row' is false, the activity succeeds, and the output will be a count of 0 rows, and an empty array.
    When the query returns one or more rows, the output with 'get only first row' returns just that first row, but when 'get only first row' is false, the output contains a count of rows, and an array of rows.
    Wednesday, June 5, 2019 6:45 PM
    Moderator
  • Hi,

    I am also stuck in a similar situation. The only way I could get the error message into web activity was to encapsulate the Web activity in a 'Execute Pipeline' activity and then passing error value as dataset parameter through the pipeline. 

    But I also need other info about the pipeline and I have tried all different ways but couldn't get it directly from the 'Copy Activity'. What did you do to get the error message ?

     

    Saturday, June 22, 2019 9:13 PM
  • Hello Raj_Hillman, the Validation activity has different output JSON depending upon what happens with the Validation activity.  This means members you might be referencing might not exist, and acessing them will throw an error.

    You mentioned Copy Activity.  Copy Activity is much better behaved than the Validation Activity.  Could you please elaborate on your situation?

    Wednesday, June 26, 2019 2:00 AM
    Moderator
  • what I would like to know is are there a UNIVERSAL way to capture the activity error message/output ?

    Up to know I can successfully capture error output from activity : Store procedure !

    Wednesday, June 26, 2019 5:41 AM
  • The way to capture information is :

    @{activity('MyActivityName').output}

    This can be used at pipeline scope, and only if there is a direct line of dependencies from that activity, to the one requesting the output.


    Here is what I have done for repro of sending validation activity output to logic app:

    The Validation activity points to a blob which does not exist.  Success or Fail, the Set Variable gets the output of the validation activity for demonstration purposes.  If the Set Variable succeeds, the Web activity sends to Logic App.  The Logic App does two things:  Write the body to a blob, and returns the body in response.

    For the Validation activity, I believe there is some confusion.  When debugging, note that there can be both an output and an 'error'.  I have circled the output and crossed the 'error'.  The 'output' is accessible.  The 'error' does not appear to be accessible.  I will need to verify with product team about the 'error'.  For this demonstration, I will be using the 'output', not the 'error'.

    For the Set Variable settings I used "@{activity('Validation1').output".  The {} convert the contents to string.  When debugging, the output of the Set Variable activity looked like: 

    {
        "name": "Display",
        "value": "{\"exists\":false,\"effectiveIntegrationRuntime\":\"DefaultIntegrationRuntime (West US)\",\"executionDuration\":31}"
    }

    For the Web activity, the settings looked like:

    I will need to finish writing later.

    Wednesday, June 26, 2019 8:01 PM
    Moderator
  • This is how I tried to ....I have tried with both i.e. passing Variable to Web Activity as well as output of the copy activity.

       
                     On Failure                On Success
    Copy Activity --------> Set Variable ----------> Web Activity
            |                                                             ^  
            |                                                             |
            |_________________On failure________________________|


    "errorMessage": "@{activity('Copy1').output.errors[0].Message}"
    Surprizingly, 'Set variable activity is able to parse the output from Copy activity but web activity fails to read the same json both the times. 

    {

    "errorMessage": "@{string(variables('var1'))}",

    "dataRead": "@{activity('Copy1').output.dataRead}", "DataFactoryName": "@{pipeline().DataFactory}", "EmailTo": "@{pipeline().parameters.EmailTo}", "PipelineName": "@{pipeline().Pipeline}", "Subject": "@{pipeline().Pipeline}", "copyDuration": "@{activity('Copy1').output.copyDuration}", "dataWritten": "@{activity('Copy1').output.dataWritten}" }


    Set Variable activity has the following code to parse the output from 'Copy1'.

    @{activity('Copy1').Error.message}

    This is the error output of the Web activity.

    {
        "errorCode": "2108",
        "message": "{\"error\":{\"code\":\"InvalidRequestContent\",\"message\":\"The request content is not valid and could not be deserialized: 'Bad JSON escape sequence: \\\\2. Path 'errorMessage', line 1, position 210.'.\"}}",
        "failureType": "UserError",
        "target": "WebActivity1"
    }






    • Edited by Raj_Hillman Thursday, June 27, 2019 3:21 PM Updated
    Thursday, June 27, 2019 3:13 PM
  • I have reproduced something very similar.  The short of it is, use @string(stuff), do not use @{stuff}.  Here are my findings:

    In the body of the web request when I use:

    {"foo":"@{variables('Display')}",
    "bar":"@string(pipeline().Pipeline)"
    }

    Then I get the response:

    {
        "errorCode": "2108",
        "message": "{\"error\":{\"code\":\"InvalidRequestContent\",\"message\":\"The request content is not valid and could not be deserialized: 'After parsing a value an unexpected character was encountered: e. Path 'foo', line 1, position 10.'.\"}}",
        "failureType": "UserError",
        "target": "Logic App"
    }

    When in the body I use:

    {"foo":"@string(variables('Display'))",
    "bar":"@string(pipeline().Pipeline)"
    }
    

    Then the Web activity succeeds.

    I believe the cause, is confusion in the parser.  The dynamic content shortcut for string @{convert-this-to-string} , is getting confused for the {} used in JSON.  This explains why @string(convert-this-to-string)  works while the other way does not.

    This is further suggested by when in the body I use:

    {"foo":"@{variables('Display')}"
    }

    and get output:

    {
        "errorCode": "2108",
        "message": "{\"error\":{\"code\":\"InvalidRequestContent\",\"message\":\"The request content is not valid and could not be deserialized: 'After parsing a value an unexpected character was encountered: e. Path 'foo', line 1, position 10.'.\"}}",
        "failureType": "UserError",
        "target": "Logic App"
    }

    While when I use:

    "foo":"@{variables('Display')}"

    Then the web activity works.  Note the difference of no enclosing {curly braces}.

    Thank you for pushing me to explore this, Raj_Hillman.  I will bring it up as a bug at next meeting.  In the meantime use @string(myStuff) .

    Monday, July 1, 2019 6:53 PM
    Moderator
  • Has there been any update with regard to this issue?
    I have a generic Send Error Web Activity set up for a number of pipelines with the request body looking like this 

    {"DataFactoryName":"@{pipeline().DataFactory}",
    "PipelineName":"@{pipeline().parameters.PipelineName}",
    "ErrorMessage":"@{pipeline().parameters.ErrorMessage}",
    "ErrorSuspect":"@{pipeline().parameters.ErrorSuspect}",
    "EmailTo":"@{pipeline().parameters.EmailTo}"}

    Then in every individual pipeline I use a parameter to retrieve the pipeline's values to be sent in the email. Such as

    PipelineName: @pipeline().Pipeline

    ErrorMessage: @{activity('Activity1').Error.Message}

    ErrorSuspect: @{json(activity('Activity1').output).pipelineName}

    EmailTo: email@domain.com

    While the request is the same for all the pipelines, one pipeline gets consistent error such as

    "The request content is not valid and could not be deserialized: 'Bad JSON escape sequence: \\D. Path 'ErrorMessage', line 3, position 82.'." while notifications for all other pipelines are being sent just fine with the same request body.

    What could be a potential issue here?



    Monday, July 15, 2019 6:51 PM
  • Hello HannaEGR.  Please try

    {"DataFactoryName":"@string(pipeline().DataFactory)",
    "PipelineName":"@string(pipeline().parameters.PipelineName)",
    "ErrorMessage":"@string(pipeline().parameters.ErrorMessage)",
    "ErrorSuspect":"@string(pipeline().parameters.ErrorSuspect)",
    "EmailTo":"@string(pipeline().parameters.EmailTo)"}

    What I suspect is happening, is the initial opening {curly brace{ sends a signal to the parser, which then assumes any subsequent {curly braces} arepart of the JSON, instead of properly recognizing it as part of the embedded dynamic expression.  This is a bug I have made product group aware of.  The work-around is to use @string(myStuff) instead of @{myStuff}.  This is not a problem when the body is just a single expression, because then there are no enclosing {curly braces}.

    Hold on, did you say that you are using the exact same body for all the pipelines, but only one  of them is throwing the error?

    Tuesday, July 16, 2019 6:25 PM
    Moderator
  • > Hold on, did you say that you are using the exact same body for all the pipelines, but only one  of them is throwing the error?

    Yep!

    I have tried using strings, but then in the email that is being sent i receive @string(pipeline().parameters.ErrorMessage) instead of the actual error message.
    Wednesday, July 17, 2019 9:29 PM
  • The error message you reported:
    "The request content is not valid and could not be deserialized: 'Bad JSON escape sequence: \\D. Path 'ErrorMessage', line 3, position 82.'."

    , compared to the line:
    "ErrorMessage":"@{pipeline().parameters.ErrorMessage}",

    clues me in on a few possibilities.  Note that there is no "D" in this line, and this line is shorter than 80 characters.  This suggests that the dynamic expression's resulting value is causing the error.  Could you please try placing a Set Variable activity  in your pipeline with the exact same expression, and debugging? I'm curious what the ErrorMessage evaluates to.

    Wednesday, July 17, 2019 11:15 PM
    Moderator
  • @HannaEGR, I have not heard back from you.  Did you find a solution on your own?
    Monday, July 22, 2019 5:42 PM
    Moderator
  • Hi Martin, thanks so much for checking in!

    I figured out why the problem arises for just one pipeline while others have no issues.
    What I have is Execute Pipeline Activity that executes a pipeline with multiple components in it. When more than one component fails in that invoked pipeline, the error arises and it is not able to retrieve error message and send email because there is more than one error message. When there is only one error in the invoked pipeline, it works fine.

    Now I need to figure out if I can find a solution to that and retrieve and send all error messages. Not sure this can be done, any tips are appreciated. Thanks!

    Monday, July 22, 2019 9:43 PM
  • That doesn't seem to match my experience.  To me it looks like only one of the two errors get bubbled up from the execute pipeline activity.
    Wednesday, July 24, 2019 9:26 PM
    Moderator
  • Hi HannaEGR, a similar issue has been escalated to the Product Group.  I believe you spoke with my colleague Kranthi in the thread entitled "Retrieve all error messages from child pipeline in Execute Pipeline Activity"

    If it is alright with you, I will stop monitoring this conversation.

      
    Friday, July 26, 2019 9:02 PM
    Moderator
  • i ran into similar problem but finally found a very easy workaround. i.e just paste your body content into the body text field box WITHOUT clicking on 'Add Dynamic Content'. and published the datafactory changes. When i trigger it ran successfully. Infact tried it worked also without converting JSON (ur case-> @{activity('Validation2').output} ) to string

    don't forget to delete the dynamic content text and then paste your below text into body text field

    {"DataFactoryName": "@{pipeline().DataFactory}","EmailTo": "email@domain.com","PipelineName": "@{pipeline().Pipeline}","ActivityEndtime": "@{string(pipeline().TriggerTime)}","ActivityErrorOutput":"@{string(activity('Validation2').output)}

    pls let me know if it worked .


    • Edited by ss_thind Saturday, August 24, 2019 11:10 PM past to paste
    Saturday, August 24, 2019 11:08 PM
  • @ss_thind are you saying that pasting the same content in the text box vs the dynamic content window results in different behavior?  Interesting.
    Tuesday, August 27, 2019 8:59 PM
    Moderator