Worker role using WCF service to download data

Question

Hi All

I'm developing a service based on Cloud. My service would have a worker role (talks to a WCF service) and the webrole. The purpose of the webrole is to request the worker role to download some data from a given FTP server. When the webrole queues it's request, the worker role checks if it has the data already or not from table. If it doesn't have the data, it requests the WCF service hosted somewhere else, to download the data for it from a given FTP server (this data can be large, around 200 MB). So, the worker role is basically acting like a client for the WCF service. For small data this works, but when the data exceeds a certain time to download(1 min), the client connection ends, so my worker role throws and exception("An error occurred while receiving the HTTP response to http://myservice.cloudapp.net/Service1.svc?wsdl. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details.") and thus I never get to know if the data has been downloaded or not, by the service. I  can see the data being stored by the WCF service in the BLOB(same for worker and service). So my question is:

1) Is there any way to increase the timeout on the WCF service side. The service is hosted from a web role, should I consider hosting the service from this webrole, and then specify the type of binding, etc from code?

This is the web.config for my WCF service (I used the VS 2010 template to create it, so, nowhere do I specify the type of binding to use or other connection parameters, except in config file)

<?xml version="1.0"?>

<configuration>

  <configSections>

  </configSections>

  <system.diagnostics>

    <trace>

      <listeners>

        <add type="Microsoft.WindowsAzure.Diagnostics.DiagnosticMonitorTraceListener, Microsoft.WindowsAzure.Diagnostics, Version=1.0.0.0, Culture=neutral, PublicKeyToken=12321fwefsd4rew"

          name="AzureDiagnostics">

          <filter type="" />

        </add>

      </listeners>

    </trace>

  </system.diagnostics>

  <system.web>

    <compilation debug="true" targetFramework="4.0" />

  </system.web>

  <system.serviceModel>

   <bindings>

      <basicHttpBinding>

        <binding name="basicHttpBinding" bypassProxyOnLocal="false" useDefaultWebProxy="true"

                 receiveTimeout="23:59:59" sendTimeout="03:10:00">

        </binding>

      </basicHttpBinding>

    </bindings>

    <behaviors>

      <serviceBehaviors>

        <behavior>

          <serviceMetadata httpGetEnabled="true"/>

          <serviceDebug includeExceptionDetailInFaults="true"/>

        </behavior>

      </serviceBehaviors>

    </behaviors>

    <serviceHostingEnvironment multipleSiteBindingsEnabled="true" />

  </system.serviceModel>

  <system.webServer>

    <modules runAllManagedModulesForAllRequests="true"/>

  </system.webServer>

</configuration>

 

2) Will Asychronous callback work to solve this? I was thinking of making the worker role wait until the WCF service downloads the data and calls it back to tell that the data has been downloaded.

3) I have used Trace.Information to trace the progress, but I can't see where the logs are on the virtual machine, anything on that?

 

Thanks!

Answer 1

You should implement a keep-alive mechanism.

See this link: http://code.msdn.microsoft.com/WCF-Azure-NetTCP-Keep-Alive-09f50fd9

Answer 2

Hi,

  >> so my worker role throws and exception("An error occurred while receiving the HTTP response to   http://myservice.cloudapp.net/Service1.svc?wsdl. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details.")

I'm a bit confused. Do you mean you queue a work item in your web role, to request a worker role to start invoke a WCF service hosted in the web role? This architecture is a bit strange. The worker role doesn't seem to have any practical usage. Please consider to use a single web role only. If you mean you have two web roles, one queues work items, and one handles web services, then it's fine.

As Sandrino points out you can implement a keep-alive mechanism. As a matter of fact Windows Azure load balancer may kill connections after 1 minute. So your solution won't work. You can also consider to use internal endpoints for communication between the web/worker role, then the communication will not involve the load balancer, and you will not hit this issue.
 
As for the tracing issue, I would like to suggest you to make sure you've configured the diagnostics monitor as described in http://msdn.microsoft.com/en-us/library/windowsazure/gg433049.aspx.

 

Best Regards,

Ming Xu.

---

Please mark the replies as answers if they help or unmark if not.
If you have any feedback about my replies, please contact [e-mail].
Microsoft One Code Framework

Answer 3

Thanks guys!

Yeah, I have two webroles, but it's just that the other webrole is not a part of where the other worker role and web role are.

 

Thanks for the suggestion of using keep-alive, I'll try that.

I already had all the settings for diagnostics, the only problem is that I am still unable to see the logs in storage(I'm using Neudesic, if that helps).

Where do I specify where the logs would be saved and what file name will be given to them?

---

RK

Answer 4

Hi,

By default only some infrastructure information is transferred to table storage. You can find it in the WADDiagnosticInfrastructureLogsTable table. If you want more data, please make sure you’ve configured to transfer them as pointed out in the above link. You will see more blob containers (such as wad-iis-logfiles) and more tables (such as WADLogsTable).

 

Best Regards,

Ming Xu.

---

Please mark the replies as answers if they help or unmark if not.
If you have any feedback about my replies, please contact [e-mail].
Microsoft One Code Framework

Answer 5

Hey I am new to azure and I have been assigned a task in which i have to use worker role to process my data request(Queue) to a  web service hosted on some other server .

plz help...