Azure Blob File Download through Azure API Management Fails with error - BackendConnectionFailure: at transfer-response RRS feed

  • Question

  • I have a requirement that should allow user to allow download file through Azure Blob Storage. I am not supposed to expose the blob storage or generate SAS for a file and expose it to end user. So for this purpose i have used API Management and in the inbound policy i am generating SAS and forming the complete URL for blob download and setting it as Backend service.

    I am able to download files with size of 14 GB through API Management with a through put of 10MBPS. But I also want to download a file that is of size 200 GB. When i try to download this file, the download is initiated and i am able to download some content but after a while it fails with below error. And during the download the max throughput achieved is 10 MBPS.

    "OpenSSL SSL_read: Connection was reset, errno 10054"

    After I check App Insight, i see following error - 

    BackendConnectionFailure: at transfer-responseUnable to read data from the transport connection: An existing connection was forcibly closed by the remote host

    It seems that this error means there was a problem at blob storage but does not exactly state what it could be.

    If i use the actual SAS url that is generated out of API Management and download the file downloads completely with a much higher throughput of 90 MBPS.

    I have not set any bandwidth limit or download limit using policy in APIM.

    I am trying to check if there is any default setting that is preventing this file to be downloaded either on Blob or on APIM? And also trying to figure out why the throughput is so low when i download the file using APIM.

    Note: I am using an Azure VM with good configuration and using curl to test my API.

    Friday, February 14, 2020 2:46 PM

All replies

  • From a quick look, it seems that the connection is breaking from Blob -> APIM, which is a ~14hr long download at 10mbps?

    "Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host" error originates from when the service that APIM is connected to becomes unavailable (note this SO post for context); and the "BackendConnectionFailure: at transfer-response" means that the connection issue occurred as APIM was sending the response to the client.

    I'll ask internally from the APIM team for any limitations, if any. In case this is something that might require a thorough investigation into your environment & configuration, does your subscription carry a support plan?

    Monday, February 17, 2020 3:12 AM
  • The download time varies between 12-14 hours but fails without full download. Yes, I have the support plan with my subscription do you suggest raising support ticket? Meanwhile, let me know if you hear anything from APIM Team. 
    Wednesday, February 19, 2020 4:35 AM