Understanding resubmittion of deadletter messages

Question

Hi

I have a question about resubmitting deadlettered messages from subscription deadletter sub queue to the same topic.

In my scenario I'm reading a message from a subscription and sending it to a end system. Since that end system is unstable an error will be thrown. I catch the error and save the message to the deadletter queue. In a GUI application an end user will be able to resubmit the deadlettered message to the same topic in order to continue from the failed step.

"InvalidOperationException: A received message cannot be directly sent to another entity. Construct a new message object instead." is thrown when I pick up the messages from the deadletter sub queue and want to resubmit it to the same topic. According to the exception text I'm now forced to to create new BrokeredMessage. So I do that but I'm allow to copy the message ID from the deadlettered message and republish it without any problems.

Can somebody explain to me why do I have to create new BrokeredMessage prior resubmitting the message to the topic and not use the original one (remember that I pick and remove the message from the subscription up prior to resubmitting) ???

Kind regards

O.S.

Answer 1

Hi Omar,

I just hit the same problem a couple of hours ago.

>>According to the exception text I'm now forced to to create new BrokeredMessage.

This is true.

>>I'm allow to copy the message ID from the deadlettered message and republish it without any problems.

Yes, you can do that, a new message ID is created when you construct the message, you can overwrite it. The MessageId is used in the duplicate detection functionality in queues and topics. I have a sample where I set my own values for this here:

http://www.cloudcasts.net/devguide/Default.aspx?id=13018

>>Can somebody explain to me why do I have to create new BrokeredMessage prior resubmitting the message to the topic and not use the original one (remember that I pick and remove the message from the subscription up prior to resubmitting) ???

I think it may be related to values in the message header. Properties like SequenceId and EnqueuedTimeUtc are set when the message is enqueued, so sending a message with these initialized will cause problems. I'm not sure if this is the true reason, I have not had time to dig into it much.

It would be great if there was a simple way to create a copy of a message, this is not simple if you also have to transfer the header values and property collection. (BrokeredMessage.Clone would be nice).

Paulo Savaori has some code in this article that shows what is involved:

http://msdn.microsoft.com/en-us/library/windowsazure/hh532261(v=vs.103).aspx

(The CreateMessageForWcfReceiver method).

I'm working on a section about cloning messages that I may add to my Service Bus e-book: http://www.cloudcasts.net/devguide/

Regards,

Alan

 

---

http://www.CloudCasts.net - Community Webcasts Powered by Azure

Answer 2

Hi Alan

Thx for the answer. Yes, I read some parts of your book couple of months ago:) It is a valuable asset for us recent Service Bus developers. Originally I'm a BizTalk developer:)

I have another question that you may help me with. Imagine my deadletter (from now on DL) sub queue containing several messages. We know that there is no way of getting DL message count, only message count for the subscription. So, I'm using a for-loop with subscription message count as iteration count and traversing through all of DL messages looking for a specific message ID. When the message ID is found I exit the loop. But is a loop the right way of picking up a single message from the DL queue or should there be created a dynamic subscription with a specific (might sound crazy) message ID filter (message ID is "promoted" in the BrokeredMessage.Properties)? Remember that I'm only after a specific message in the DL queue so I'am using ReceiveMode.PeekLock-mode and then calling Complete() when the message is found.

I guess MessageReceiver.Receive jumps to the next message in the DL queue because I never call BrokeredMessage.Abanndon() while traversing through the loop so all traversed messages are locked until their lock timeout expires. The only examples of traversing the DL queue on the Internet are by using the foor-loop and using ReceiveMode.ReceiveAndDelete. What are your thoughts on this?

Kind regards

Omar

Answer 3

Hi,

>>We know that there is no way of getting DL message count, only message count for the subscription.

Yes, it would be nice to have DeadletterMessageCount in the description classes.

>>But is a loop the right way of picking up a single message from the DL queue

I can't think of another way right now. One issue you will hit is if you have more than one application tryting to retrieve messages, as the message that the second application may be looking for may still be hidden from the first. Is there a way you can abandon the other messages once you have received the one you are looking for?

Another issue is what will happen if there are a few thousand messages in the DL queue. Will you be billed for thousands of message operations to retrieve one message?

Maybe a better option is to dequeue the DL messages to another storage system, maybe Azure blobs or tables, or something on-prem. That way you can easily locate the ones you want.

Please let me know if you think of something better...

Regards,

Alan

 

---

http://www.CloudCasts.net - Community Webcasts Powered by Azure

Answer 4

Hi again,

I just had a thought. If you can defer messages in the DLQ you could receive then defer the messages and store the MessageId and SequenceId. When you want to receive the message you can then receive it using the SequenceId. I have not tried deferring messages on a DLQ though, it may work...

Alan

---

http://www.CloudCasts.net - Community Webcasts Powered by Azure

Answer 5

Messages that come out of Service Bus may have system properties set that you may not really want to have set when you resubmit the message, so that would be a splendid source of obscure bugs. That said, I agree we should have a CopyFrom() or CopyTo() function or a factory function that can take an existing message. Until we have that, that seems like a simple utility function to have sitting around somewhere.

Answer 6

Hi Allan

>>Maybe a better option is to dequeue the DL messages to another storage system, maybe Azure blobs or tables, or something on-prem. That way you can easily locate the ones you want.

Just what I was thinking. I think that storing failed messages in table is a much better option. Looping through the DLQ to find a specific message does not sound very Pub-Sub'ish:) We should maybe threat DLQ as any other queue, i.e. you always dequeue the first message, regardless of what it is.

So, my question is how do people handle failed messages messaging systems (Service Bus, MSMQ), what is a DLQ strategy? A DLQ can potentially store multiple and various messages. Should you always consume the first one or should you traverse DLQ looking for a specific message????

Kind regards

Omar

Answer 7

Bump:)

Answer 8

Hi Omar,

You got me thinking now...

I am working on an example of this for my e-book. I plan to implement a "Wiretap" pattern that will allow a doagnostics application to monitor messages on a diagnostics subscription. The same techniques could be used for monitoring a dead.letter queue as well. I plan to do something like this:

Useing a simple WPF application.

•Grab about 20 messages and show details in a ListView.
•Give the user the option to deserialize and inspect the messages.
•Give the user the option to complete or abandon the messages.
I'll blog and maybe webcast on it when I get it working (should be this week sometime).

Alan

---

http://www.CloudCasts.net - Community Webcasts Powered by Azure

Answer 9

Hi Allan

Thx for the answer. I'll follow your blog during the next days:)

Omar

Answer 10

Hi,

Yes. There is a Clone method on BrokeredMessage in the latest SDK.

http://msdn.microsoft.com/en-us/library/microsoft.servicebus.messaging.brokeredmessage.clone.aspx

Regards,

Alan

---

Free e-book: Windows Azure Service Bus Developer Guide.

Answer 11

With regards to the "CopyFrom() or CopyTo()" feature request suggested by clemensv, can I use the BrokeredMessage Clone method to do this?

Can anyone forsee any dangers or pitfalls in taking a clone of a BrokeredMessage from the deadletter queue a resubmitting it to the main queue?

I've tried it and it seems to work fine.