Getting this error from service bus every now and then.

Question

We are getting random errors in our production environment. Everything works just fine most of the time, but a couple of times a day we get an error like this one.

Microsoft.ServiceBus.Messaging.MessagingException: The server was unable to process the request; please retry the operation. If the problem persists, please contact your Service Bus administrator and provide the tracking id..TrackingId:********-e7cf-4e95-b966-fa2d859ac5cf_G27_B6,TimeStamp:2/12/2013 1:51:17 AM ---> System.ServiceModel.FaultException`1[System.ServiceModel.ExceptionDetail]: The server was unable to process the request; please retry the operation. If the problem persists, please contact your Service Bus administrator and provide the tracking id..TrackingId:********-e7cf-4e95-b966-fa2d859ac5cf_G27_B6,TimeStamp:2/12/2013 1:51:17 AM

Server stack trace: 
   at Microsoft.ServiceBus.Messaging.Sbmp.DuplexRequestBindingElement.DuplexRequestSessionChannel.ThrowIfFaultMessage(Message wcfMessage)
   at Microsoft.ServiceBus.Messaging.Sbmp.DuplexRequestBindingElement.DuplexRequestSessionChannel.HandleMessageReceived(IAsyncResult result)

Exception rethrown at [0]: 
   at Microsoft.ServiceBus.Common.AsyncResult.End(IAsyncResult result)
   at Microsoft.ServiceBus.Messaging.Sbmp.DuplexRequestBindingElement.DuplexRequestSessionChannel.DuplexCorrelationAsyncResult.End(IAsyncResult result)
   at Microsoft.ServiceBus.Messaging.Channels.ReconnectBindingElement.ReconnectChannelFactory`1.RequestSessionChannel.RequestAsyncResult.<GetAsyncSteps>b__13(RequestAsyncResult thisPtr, IAsyncResult r)
   at Microsoft.ServiceBus.Messaging.IteratorAsyncResult`1.StepCallback(IAsyncResult result)

Exception rethrown at [1]: 
   at Microsoft.ServiceBus.Common.AsyncResult.End(IAsyncResult result)
   at Microsoft.ServiceBus.Messaging.Channels.ReconnectBindingElement.ReconnectChannelFactory`1.RequestSessionChannel.EndRequest(IAsyncResult result)
   at Microsoft.ServiceBus.Messaging.Sbmp.SbmpMessageReceiver.EndReceiveCommand(IAsyncResult result, ref IEnumerable`1 messages)
   --- End of inner exception stack trace ---

Server stack trace: 
   at Microsoft.ServiceBus.Messaging.Sbmp.SbmpMessageReceiver.EndReceiveCommand(IAsyncResult result, ref IEnumerable`1 messages)
   at Microsoft.ServiceBus.Messaging.IteratorAsyncResult`1.StepCallback(IAsyncResult result)

Exception rethrown at [0]: 
   at Microsoft.ServiceBus.Common.AsyncResult.End(IAsyncResult result)
   at Microsoft.ServiceBus.Messaging.Sbmp.SbmpMessageReceiver.OnEndTryReceive(IAsyncResult result, ref IEnumerable`1 messages)
   at Microsoft.ServiceBus.Messaging.MessageReceiver.TryReceive(TrackingContext trackingContext, Int32 messageCount, TimeSpan timeout, ref IEnumerable`1 messages)
   at Microsoft.ServiceBus.Messaging.MessageReceiver.Receive(TimeSpan serverWaitTime)
   at Microsoft.Practices.TransientFaultHandling.RetryPolicy.ExecuteAction(Func`1 func)
   at IQ.IR.Core.ServiceBus.AzureBus`1.Listen(IMessageProcessor`1 messageProcessor) in AzureBus.cs: line 108; TraceSource 'WaWorkerHost.exe' event

The failing line of code :

var receivedMessage = _retryPolicy.ExecuteAction(() => _messageReceiver.Receive(TimeSpan.FromMinutes(15)));

We are using Microsoft.Practices.TransientFaultHandling and the above retry policy is:

RetryPolicy<ServiceBusTransientErrorDetectionStrategy>(new ExponentialBackoff())

who the heck is our Service Bus administrator? And what does this error mean?

---

woaksie

Answer 1

HI

I had answered this thread:

http://social.msdn.microsoft.com/Forums/en-US/windowsazureconnectivity/thread/728a5c55-e91a-440d-b325-516a4100d3b9/

Which I think you may got the same problem.

---

Dino He
MSDN Community Support | Feedback to us
Develop and promote your apps in Windows Store
Please remember to mark the replies as answers if they help and unmark them if they provide no help.

Answer 2

That other thread sounds like what we are experiencing but contains no solution or valid explanation other than Service Bus is flaky. Still need this to be fixed.

---

woaksie

Answer 3

HI

That's course by Azure, you can check the state on dashboard:

http://www.windowsazure.com/en-us/support/service-dashboard/

And your problem should be fiexed now.

Answer 4

During the period between Dec 14th 2012 and Feb 13th 2013 we saw 3,353 of these exceptions in our logs. The service dashboard showed Service Bus to be normal for nearly all of that time.

The dashboard in our experience does not show a lot of problems, even after MS acknowledge there has been a problem.

---

woaksie