locked
Azure IoT Hub -> Service Bus Topic | Ordered Messages RRS feed

  • Question

  • Hi community,

    I am currently trying to achieve the following:

    Our devices are sending telemetry data to the Azure IoT Hub. For our backend processing it is nescessary to keep the order of the received messages per devices. Our approach is to receive it via the Azure IoT Hub and then use an Service Bus Topic Endpoint:

    Device -> Azure IoT Hub -> Routing -> Endpoint "Service Bus Topic"

    At the Topic we have several subscriptions which were consumed by Azure Functions. Now we would like to keep the order of the received messages per device.

    For that requirement we tried to set the DeviceId as SessionId on the BrokeredMessage. But in Azure Iot Hub Message you cannot set the SessionId. Even after some research if found "Service Bus queues and topics with Sessions or Duplicate Detection enabled are not supported as custom endpoints." at https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-devguide-messages-read-custom.

    How can we asure ordered processing? Is the approach state above wrong?

    Thanks for your thoughts and comments!

    Friday, June 29, 2018 10:44 AM

All replies

  • Hello DanielVSTO,

    From this official document we can see that:

    IoT Hub message routing maintains the order in which messages are sent, so that they arrive in the same way.

    So the endpoint(here Service Bus Topic for you) will receive events in the same order that they occurred.

    So feel free to use and post here if you any issue about this.

    Best regards,

    Rita


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Monday, July 2, 2018 1:58 AM
  • Hello DanielVSTO,

    From this official document we can see that:

    IoT Hub message routing maintains the order in which messages are sent, so that they arrive in the same way.

    So the endpoint(here Service Bus Topic for you) will receive events in the same order that they occurred.


    Hello Rita,

    Thank you for your reply and the quote from the documentation!

    Our issue is that the order of the messages is only kept if there is one consuming process (in our case Azure Function). But we like to consume the messages with several parallel processes (Azure Functions).

    Within these processes it is nescessary that only one process reads the messages of one device and no other process consume the next messages of the same device at the same time.

    As far as I understand this requirement is implementend in the Azure SB Topics with the SessionId (to support parallel/ordered consumption of messages).

    But unfortunetaly there is not way to set this SessionId within the Azure Iot Hub?

    Best regards,

    Daniel

    Wednesday, July 4, 2018 9:14 AM
  • Hello DanielVSTO,

    What I think about are two solutions:

    One is checking the device id in azure function, different function only select one device messages to consume. The disadvantage is all messages go to every function and need a time to check the device id. But no need service bus. You can use Azure IoT Hub(Event Hub) to trigger the function.

    Second is using event hub to receive these messages and send to service bus topic?

    About Azure IoT Hub endpoint support service bus session you can submit a feature question here.

    Best regards,

    Rita


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Thursday, July 5, 2018 8:17 AM
  • Hey,

    Today we have some new features that can enable you the scenario described:

    https://docs.microsoft.com/en-us/azure/iot-hub/iot-hub-message-enrichments-overview

    "Message enrichments is the ability of the IoT Hub to stamp messages with additional information before the messages are sent to the designated endpoint. One reason to use message enrichments is to include data that can be used to simplify downstream processing."

    Thanks!

    Tuesday, October 15, 2019 5:03 PM
  • Hi 

    This feature can not resolve the problem, because all additional properties will come to the custom properties collection. To resolve the problem SessionId property should come to System Properties collection, because service bus will not take into account custom properties in this case.

    I just created a proposal to implement it: Add system property SessionId to the Message


    • Edited by DVladimir Friday, November 8, 2019 3:05 PM
    Friday, November 8, 2019 3:04 PM
  • I just found, that there is a possibility to assign a value to sys.sessionId property in a azure service bus topic subscription rule.
    Just define following SqlRuleAction: set sys.sessionId="iothub-connection-device-id"
    But I also found, that it does not work with my already existed topics, I had to create a new one.
    Saturday, November 9, 2019 2:44 PM