Azure Queue - Messages get stuck

Question

I am experiencing an issue where one or more messages are getting "stuck" in the queue on Azure.  I'm pulling the messages off using QueueClient.Receive.  What happens is that the Azure queue shows a count of > 0, but the .Receive method results in a null Brokered Message.  No errors are thrown, it's just that there are messages to receive, but the .Receive doesn't see them.  I'm getting nowhere fast, and I'm desperate to have this working.  Letting my queue watcher run all night, there are two error that I saw that don't mean anything to me, and that possibly have to do with the messages not being pulled.  The errors are below.  Thank you for any assistance you can provide, and please let me know what other information you need.  

Exception 1:

Microsoft.ServiceBus.Messaging.MessagingException: The server was unable to process the request due to an internal error..TrackingId:06fe7bc6-a891-4df8-8bae-f2d2867337d8_12_6,TimeStamp:11/18/2011 7:27:59 AM ---> System.ServiceModel.FaultException`1[System.ServiceModel.ExceptionDetail]: The server was unable to process the request due to an internal error..TrackingId:06fe7bc6-a891-4df8-8bae-f2d2867337d8_12_6,TimeStamp:11/18/2011 7:27:59 AM

 

Server stack trace:

   at Microsoft.ServiceBus.Messaging.Sbmp.DuplexRequestBindingElement.DuplexRequestSessionChannel.ThrowIfFaultMessage(Message wcfMessage)

   at Microsoft.ServiceBus.Messaging.Sbmp.DuplexRequestBindingElement.DuplexRequestSessionChannel.HandleMessageReceived(IAsyncResult result)

 

Exception rethrown at [0]:

   at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)

   at Microsoft.ServiceBus.Messaging.Sbmp.DuplexRequestBindingElement.DuplexRequestSessionChannel.EndRequest(IAsyncResult result)

   at Microsoft.ServiceBus.Messaging.Channels.ReconnectBindingElement.ReconnectChannelFactory`1.RequestSessionChannel.RequestAsyncResult.<GetAsyncSteps>b__13(RequestAsyncResult thisPtr, IAsyncResult r)

   at Microsoft.ServiceBus.Messaging.IteratorAsyncResult`1.StepCallback(IAsyncResult result)

 

Exception rethrown at [1]:

   at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)

   at Microsoft.ServiceBus.Messaging.IteratorAsyncResult`1.End(IAsyncResult asyncResult)

   at Microsoft.ServiceBus.Messaging.Channels.ReconnectBindingElement.ReconnectChannelFactory`1.RequestSessionChannel.EndRequest(IAsyncResult result)

   at Microsoft.ServiceBus.Messaging.Sbmp.SbmpMessageReceiver.EndReceiveCommand(IAsyncResult result, IEnumerable`1& messages)

   --- End of inner exception stack trace ---

 

Server stack trace:

   at Microsoft.ServiceBus.Messaging.Sbmp.SbmpMessageReceiver.EndReceiveCommand(IAsyncResult result, IEnumerable`1& messages)

   at Microsoft.ServiceBus.Messaging.Sbmp.SbmpMessageReceiver.ReceiveAsyncResult.<GetAsyncSteps>b__35(ReceiveAsyncResult thisPtr, IAsyncResult a)

   at Microsoft.ServiceBus.Messaging.IteratorAsyncResult`1.StepCallback(IAsyncResult result)

 

Exception rethrown at [0]:

   at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)

   at Microsoft.ServiceBus.Messaging.Sbmp.SbmpMessageReceiver.OnEndTryReceive(IAsyncResult result, IEnumerable`1& messages)

   at Microsoft.ServiceBus.Messaging.MessageReceiver.OnTryReceive(TrackingContext trackingContext, Int32 messageCount, TimeSpan timeout, IEnumerable`1& messages)

   at Microsoft.ServiceBus.Messaging.MessageReceiver.TryReceive(TrackingContext trackingContext, Int32 messageCount, TimeSpan timeout, IEnumerable`1& messages)

   at Microsoft.ServiceBus.Messaging.MessageReceiver.TryReceive(TimeSpan serverWaitTime, BrokeredMessage& message)

   at Microsoft.ServiceBus.Messaging.QueueClient.Receive(TimeSpan serverWaitTime)

   at Uti.LMS.AppEventProcessor.LmsAppEventProcessor.StartProcessing() in

 

Exception 2: 

Microsoft.ServiceBus.Messaging.MessagingException: Internal Server Error: The server did not provide a meaningful reply; this might be caused by a premature session shutdown..TrackingId:dd9c3de3-ff2a-407c-a1c0-d1af15b0064b, Timestamp:11/18/2011 9:58:09 AM ---> System.ServiceModel.CommunicationException: Internal Server Error: The server did not provide a meaningful reply; this might be caused by a premature session shutdown..TrackingId:dd9c3de3-ff2a-407c-a1c0-d1af15b0064b, Timestamp:11/18/2011 9:58:09 AM

 

Server stack trace:

 

 

Exception rethrown at [0]:

   at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)

   at Microsoft.ServiceBus.Messaging.Sbmp.DuplexRequestBindingElement.DuplexRequestSessionChannel.EndRequest(IAsyncResult result)

   at Microsoft.ServiceBus.Messaging.Channels.ReconnectBindingElement.ReconnectChannelFactory`1.RequestSessionChannel.RequestAsyncResult.<GetAsyncSteps>b__13(RequestAsyncResult thisPtr, IAsyncResult r)

   at Microsoft.ServiceBus.Messaging.IteratorAsyncResult`1.StepCallback(IAsyncResult result)

 

Exception rethrown at [1]:

   at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)

   at Microsoft.ServiceBus.Messaging.IteratorAsyncResult`1.End(IAsyncResult asyncResult)

   at Microsoft.ServiceBus.Messaging.Channels.ReconnectBindingElement.ReconnectChannelFactory`1.RequestSessionChannel.EndRequest(IAsyncResult result)

   at Microsoft.ServiceBus.Messaging.Sbmp.SbmpMessageReceiver.EndReceiveCommand(IAsyncResult result, IEnumerable`1& messages)

   --- End of inner exception stack trace ---

 

Server stack trace:

   at Microsoft.ServiceBus.Messaging.Sbmp.SbmpMessageReceiver.EndReceiveCommand(IAsyncResult result, IEnumerable`1& messages)

   at Microsoft.ServiceBus.Messaging.Sbmp.SbmpMessageReceiver.ReceiveAsyncResult.<GetAsyncSteps>b__35(ReceiveAsyncResult thisPtr, IAsyncResult a)

   at Microsoft.ServiceBus.Messaging.IteratorAsyncResult`1.StepCallback(IAsyncResult result)

 

Exception rethrown at [0]:

   at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)

   at Microsoft.ServiceBus.Messaging.Sbmp.SbmpMessageReceiver.OnEndTryReceive(IAsyncResult result, IEnumerable`1& messages)

   at Microsoft.ServiceBus.Messaging.MessageReceiver.OnTryReceive(TrackingContext trackingContext, Int32 messageCount, TimeSpan timeout, IEnumerable`1& messages)

   at Microsoft.ServiceBus.Messaging.MessageReceiver.TryReceive(TrackingContext trackingContext, Int32 messageCount, TimeSpan timeout, IEnumerable`1& messages)

   at Microsoft.ServiceBus.Messaging.MessageReceiver.TryReceive(TimeSpan serverWaitTime, BrokeredMessage& message)

   at Microsoft.ServiceBus.Messaging.QueueClient.Receive(TimeSpan serverWaitTime)

   at Uti.LMS.AppEventProcessor.LmsAppEventProcessor.StartProcessing() in 

Answer 1

Hi,

Are you experiencing this issue for a particular message or all messages in the queue? If you have problems with any messages, please check your code to see if there is anything wrong. You can also post the code here so we can have a look. If it's a particular message, please try to retrieve it again after a few seconds. If you still can't retrieve the message, you can contact our customer support on https://support.microsoft.com/oas/default.aspx?gprid=14924&st=1&wfxredirect=1&sd=gn, and provide the tracking ID obtained from the response. Our support team will look into the request to find the issue.

 

Best Regards,

Ming Xu.

---

Please mark the replies as answers if they help or unmark if not.
If you have any feedback about my replies, please contact [e-mail].
Microsoft One Code Framework

Answer 2

It is not for a particular message. Every so often, one or more messages get "stuck", and the .Receive method returns null as if no messages exist.

Answer 3

I submitted a support ticket, ticket ID: 111112160899854

 

Thanks,

Neil

Answer 4

Could anybody please update the progress on this? we are also facing similar issue.

-Sachin Sancheti

Answer 5

almost one year, the service bus still doesn't work properly? I had problem on one existing Topic, could not get it work, but when I just created a new Topic, my code worked fine.

And I still get problem on sending message, the worst thing is the topicClient.Send(message) returns without an exception, but the message disappears in nowhere.

Answer 6

Hi,

Sorry to hear about the issue you are seeing with Service Bus. Regarding the send behavior we will always throw an exception if the message is not successfully submitted to the topic. Can you please check the following:

1) For the subscriptions that you are receiving from, what are the filters set? You can do this from Visual Studio using server explorer or the Service Bus Explorer tool

2) Is Deadlettering behavior enabled for the Topic? If there are errors in filter evaluation or if the message is peek locked more than the max delivery count then the message will be moved to the dead lettered queue

3) From the receiver, are you using ReceiveAndDelete or PeekLock mode?

---

Regards, -Abhishek Lal
Senior Program Manager
Windows Azure

Answer 7

Same thing is happening with me. I am using ReceiveAndDelete mode to receive from a topic subscription. Active message count is > 0, DeadLetter count = 0. Message delivery count = 2. Also, the topic shows an active message count of 0, while the subscription shows an active message count > 0. After updating the subscription, the active message count returned to 0. I rely on the subscription message count to determine the health of my app, so this is a little frustrating.

Answer 8

We do provide counts on Queues/Topics/Subscriptions and include sub-queue counts (like scheduled, dead-lettered etc.) for monitoring the state for these items. These counts are inherently changing when the application is running so they should be used appropriately. Active message count should tell you how many messages you can get from the Subscription. Not sure message delivery count is referring to. Also we have recently added more metric thru the Azure portal to see details of calls/errors against your topics/subscriptions.

---

Regards, -Abhishek Lal
Senior Program Manager
Windows Azure