locked
Azure Queue Performance Problem RRS feed

  • Question

  • I have a Web and a Worker Role which should communicate with low latency.

    The Workflow: Web Role adds a request to a queue ("requestqueue"), the Worker Role is polling the requestqueue and process the request (does some work). After finishing with the work, the Worker send a response message to another queue ("responsqueue"), the responsqueue is polled by the web role.

    My problem is that this communication is quite slow: i takes 2-30 seconds (the actual work oft the Worker Role not included).

    I found an related topic http://social.msdn.microsoft.com/Forums/en-US/windowsazure/thread/474ee03f-f2ad-4031-8924-bebf61655536 , but it's more than 2 years old and non of the replies helped. Putting the functionality of the worker role in the web role is no possibilty for my project.

    Is the Azure Queue the wrong approach for fast communication between Web and Worker Role ? Should i use the Service Bus for the communication or are there other alternatives? 

     

    Monday, January 17, 2011 12:47 PM

Answers

  • How fast are you going to poll the queue?  It seems you're always going to have a second or two round-trip latency just because of polling frequency.

    If you need low-latency communication, I think you'll have to communicate directly.  (Use a WCF service, raw sockets, or the like.)

    I don't quite understand what you mean by "I think this wouldn't work, because the Worker Role in my project is stateful."

    • Marked as answer by Robar666 Tuesday, January 18, 2011 3:45 PM
    Monday, January 17, 2011 5:32 PM
  • Hi

    sending is not the problem but thanks anyway. Also thx to Alan Smith and Steve Marx they brought up the idea of using a WCF service. I've now replaced the queue communication with a WCF service, it works already (nice low latency), but at the moment only synchronously, i will fix this later so i can make asynch calls.

    • Marked as answer by Robar666 Tuesday, January 18, 2011 3:45 PM
    Tuesday, January 18, 2011 3:45 PM

All replies

  • Hi,

    You could expose the functionality as a self-hosted WCF service in the worker role and call it synchronously, though this is almost the same as putting the functionlaity in a web role.

    Is it possible to have a second web role that could host a service?

    Regards,

    Alan

     


    http://www.CloudCasts.net - Community Webcasts Powered by Azure
    Monday, January 17, 2011 2:51 PM
  • Hi Alan,

    I think this wouldn't work, because the Worker Role in my project is stateful. So i can't figure a way out, how this could work with a WCF service.

    And what do you mean "a second web role that could host a service?"

     

    Regards, 

    Patrick

    Monday, January 17, 2011 3:16 PM
  • Hi,

    I was thinkgin to have one role for the web site, and another for web services. If your worker role is stateful it may not be an option.

    Regards,

    Alan

     


    http://www.CloudCasts.net - Community Webcasts Powered by Azure
    Monday, January 17, 2011 3:38 PM
  • How fast are you going to poll the queue?  It seems you're always going to have a second or two round-trip latency just because of polling frequency.

    If you need low-latency communication, I think you'll have to communicate directly.  (Use a WCF service, raw sockets, or the like.)

    I don't quite understand what you mean by "I think this wouldn't work, because the Worker Role in my project is stateful."

    • Marked as answer by Robar666 Tuesday, January 18, 2011 3:45 PM
    Monday, January 17, 2011 5:32 PM
  • Hi, polling time is currently 10ms. I encapsulated the functionality of the queue in two classes "CommunicationHelper" and "CommunicationHelperFactory" this classes will be used by the Web and Worker Role maybe this is the bottleneck:

     

    using System.Collections.Generic;
    using Microsoft.WindowsAzure;
    using Microsoft.WindowsAzure.StorageClient;
    using System.Threading;
    using System.Diagnostics;
    using System;
    using System.Threading.Tasks;
    
    namespace xxxxx.CommunicationHelper {
    
    	public delegate void MessageReceivedEvent(String msg);
    
    	public static class CommunicationHelperFactory {
    		private static IDictionary<string, CommunicationHelper> queueHelperMap = new Dictionary<string, CommunicationHelper>();
    
    		public static CommunicationHelper GetInstanceForName(string queueName) {
    			if (queueHelperMap.ContainsKey(queueName)) {
    				return queueHelperMap[queueName];
    			} else {
    				CommunicationHelper qh = new CommunicationHelper(queueName);
    				queueHelperMap.Add(queueName, qh);
    				return qh;
    			}
    		}
    	}
    
    	public class CommunicationHelper {
    		public event MessageReceivedEvent MessageReceived;
    		private CloudQueue queue;
    		private bool cancel = false;
    		private int pollingTime = 10;
    		private Object sendMessageLock = new Object();
    
    		internal CommunicationHelper(string helperName) {			
    			CloudStorageAccount csa = CloudStorageAccount.DevelopmentStorageAccount; //BMK change login credentials
    			var cqc = csa.CreateCloudQueueClient();
    			queue = cqc.GetQueueReference(helperName);
    
    			if (!queue.Exists())
    				queue.Create();
    
    			Task.Factory.StartNew(delegate { Run(); });
    		}
    
    		private void Run() {			
    			TimeSpan ts = new TimeSpan(0, 0, 10); // 10 seconds timeout
    			while (cancel != true) {
    				var msg = queue.GetMessage(ts);
    				if (msg != null && MessageReceived != null) {
    					MessageReceived(msg.AsString); // fire event
    					queue.DeleteMessage(msg);
    				} else { 
    					Thread.Sleep(pollingTime);
    				}
    			}
    		}
    		
    		private void SendQueueMessage(CloudQueueMessage queueMessage) {
    			lock (sendMessageLock) {
    				queue.AddMessage(queueMessage);
    			}
    		}
    		
    		public void SendMessage(string queueMessage) {
    			CloudQueueMessage msg = new Microsoft.WindowsAzure.StorageClient.CloudQueueMessage(queueMessage);
    			this.SendQueueMessage(msg);
    		}
    	}
    }
    
    
    I think i will take a look at WCF service and the service bus and check out if this works for my project. Actually, as i mentioned in my first post, i only need a fast way to communicate between Web and Worker role, so i'm open for any suggestions.

     

    Tuesday, January 18, 2011 8:14 AM
  • Hi,

    Perhaps I misunderstood the exact problem described, but if the issue is that it takes too long to "send" a message because it ties your primary thread, what about spinning a thread that sends the message to the queue (such as QueueUserWorkItem or similar)? This way your primary thread won't have to wait 2-30 seconds; it will return immediately.

     

     

     

     


    Herve Roggero, Blue Syntax MVP SQL Azure Co-Author: Pro SQL Azure
    Tuesday, January 18, 2011 2:53 PM
  • Hi

    sending is not the problem but thanks anyway. Also thx to Alan Smith and Steve Marx they brought up the idea of using a WCF service. I've now replaced the queue communication with a WCF service, it works already (nice low latency), but at the moment only synchronously, i will fix this later so i can make asynch calls.

    • Marked as answer by Robar666 Tuesday, January 18, 2011 3:45 PM
    Tuesday, January 18, 2011 3:45 PM