How to make that more streams wait each other?


  • First of all sorry for my bad english.

    This is my scenario: I have four different stream with different event payload, but there is a payload field in common to all of them, say ID.

    I want that the query release an output only when, on the four stream, there are four event, one for each stream, with all the ID values identical.

    Also I don't know the order of arrival of the four events and the period lenght between them!

    How to do this?? I try to use a join between the streams but without any good result..

    Thanks in advance!

    giovedì 12 aprile 2012 20:15


  • Hi Gabriele,

    Let's start with explaining how joins work with events. When you join between streams, only those events that overlap temporally will join. So, if you have a stream of point events, they would have to have the exact same start time - down to the tick - to join. If you have intervals, then they will join if the intervals overlap and the resulting event will have a start time and end time that is the complete overlap of all events.

    Then ... how do different streams from different source synchronize? Well, they will synchronize on CTI ... everything runs on CTIs in StreamInsight. And, unless you override the AdvanceTimeSettings to import CTIs from a different stream, they will synchronize to the slowest stream. So ... faster streams will "buffer" until all streams enqueue CTIs for a specific time and then the query will "move" forward.

    Now ... I'd have to think about it to see if your requirement could be done in a query BUT ... I do know that you can do this in a User-Defined Stream Operator (UDSO). You would union all 4 streams together and then pass to the UDSO. The UDSO would "watch" the stream and collect the values as they pass through. Since it can keep state, you would store and hold all of the values that have passed through. You would have to have a field in the payload to indicate which stream it's coming from ... so the UDSO will "know" when all 4 streams have enqueued that particular item. Once they all do, the UDSO can then release your event.

    I'm not sure if this completely explains it. I will try ... if I have time ... to get a sample for you but I may not be able to do this until Sunday or Monday unfortunately.

    DevBiker (aka J Sawyer)
    Microsoft MVP - Sql Server (StreamInsight)

    Ruminations of

    If I answered your question, please mark as answer.
    If my post was helpful, please mark as helpful.

    giovedì 12 aprile 2012 22:45