locked
Azure Function in Data Factory- Error calling the endpoint RRS feed

  • Question

  • Hi,

    I've been experimenting with Azure Functions within Data Factory. Testing has had mixed results. The Azure Function gets called and is doing what is supposed to do (creates a file) but Data Factory keeps getting an error back. 

    { "errorCode": "3600", "message": "Error calling the endpoint.", "failureType": "UserError", "target": "Azure Function1" }

    I'm not sure what to make of this since the AF runs successfully. I checked the logs in app insight and everything appears to be completely normal. The file gets created exactly as I expected it.

    Has anyone else had a similar experience? 

    I can post more information if it would help.

    Tuesday, January 29, 2019 2:31 AM

All replies

  • The return type of the Azure function has to be a valid JObject. Anything else fails and raises the generic user error “Error calling endpoint”. I suggest you try changing the return type.

    Wednesday, January 30, 2019 9:56 AM
  • That's probably it. I'll check into that and provide an update.

    Thanks for the reply!


    Thursday, January 31, 2019 12:56 PM
  • Here is a small example on how to change that:

    https://microsoft-bitools.blogspot.com/2019/01/introducing-azure-function-activity-to.html


    Please mark the post as answered if it answers your question | My SSIS Blog: http://microsoft-ssis.blogspot.com | Twitter

    Thursday, January 31, 2019 9:54 PM
  • I've had the same problem that Owen mentioned. I think it could be a greater problem because my function sometimes takes a long time (5 min aprox) and it failed in the data factory (in the monitor of the azure functions it shows an ok run), but when the run is shorter than 3.8 minutes I have no problem and get the json response correctly, and the pipeline run shows success. How could this be possible? My function runs with python and my response is:

        # response = open(os.environ['res'], 'w')
        # string_answer = {'out': 'success'}
        # response.write(json.dumps(string_answer))
        # response.close()

    Monday, February 25, 2019 2:59 PM
  • I've had the same problem that Owen mentioned. I think it could be a greater problem because my function sometimes takes a long time (5 min aprox) and it failed in the data factory (in the monitor of the azure functions it shows an ok run), but when the run is shorter than 3.8 minutes I have no problem and get the json response correctly, and the pipeline run shows success. How could this be possible? My function runs with python and my response is:

        # response = open(os.environ['res'], 'w')
        # string_answer = {'out': 'success'}
        # response.write(json.dumps(string_answer))
        # response.close()

    I'm confused as to what you are asking / saying.
    Are you asking why Data Factory reports a failure when the function takes time > 5 minutes but reports seccess with time < 3.8 minutes,
    or
    are you asking how a simple python script could take so long?

    When you set up the Function activity in Data Factory, there are options for timeout and retry intervals.

    Monday, February 25, 2019 7:49 PM
  • the first one, I am asking why Data Factory reports a failure when it takes longer than 3.8 minutes, even when my function has run successfully. And the timeout setting by default shows 7 days, I have changed this to 1 hour, or 10 minutes, or 15 minutes and the Data Factory continues reporting a failure.
    Tuesday, February 26, 2019 11:51 AM
  • I will begin reproducing your issue.  I have a few ideas about why you have that issue, but nothing definite yet.  I will let you know when I have more information.
    Tuesday, February 26, 2019 9:15 PM
  • I worked internally with the team and determined that the timeout is a software limitation you can do nothing about.  However, I do have a potential workaround.

    Your current implementation is similar to a synchronous function call.  Data factory calls the function app, and then waits for the response.  If something happens to the function app, or the response never comes, then Data factory reports a failure.

    I propose changing to something similar to an asynchronous function call.  Split your current pipeline into two pipelines, and have the function app output to blob storage instead of trying to return the data directly to Data Factory.  The first pipeline kicks off the function app.  The second pipeline uses an event trigger (is triggered by the function app write to blob storage), and then processes data output by the function app to blob storage.

    Thursday, March 7, 2019 7:28 AM
  • thank you for your detailed answer, I'll definitely follow your recommendation
    Tuesday, March 12, 2019 6:15 PM
  • Thank you.  Let me know how it works out.  If you have success, please mark as answered.
    Tuesday, March 12, 2019 11:20 PM