none
Merging two streams and finding adjacent events

    Soru

  • Hi,

    I have two streams: str1 and str2. Both of them contain similar set of data. Now with str1 as the reference stream, I would like to find events from str2 which occur just after each event in str1. For example:

    str1:

    2012/05/02 10:00:00

    2012/05/02 10:10:00

    2012/05/02 10:17:00

    str2:

    2012/05/02 10:00:02

    2012/05/02 10:01:00

    2012/05/02 10:02:00

    2012/05/02 10:10:30

    2012/05/02 10:18:00

    2012/05/02 10:20:00

    required output:

    2012/05/02 10:00:02

    2012/05/02 10:10:30

    2012/05/02 10:18:00

    I tried by altering the duration of events of str1 to TimeSpan.Max, but this returns all str2 events which occur after an event in str1. What I want is only the first event of str2 occurring after an str1 event.

    Regards,

    02 Mayıs 2012 Çarşamba 11:42

Yanıtlar

  • Here's how I got this working in LinqPad.

    1. Extend and clip the reference stream to the data stream. This will convert the reference stream to a signal whose events start at the original event start time while the end time is set to just before the start time of a matching event in the data stream.
    2. Now shift the event time of the events in the reference stream by 1 tick and join with the data stream. This will cause the events in the reference stream to overlap with the events in the data stream.

    Here's the code:

    var referenceStream = from e in str1
    		.AlterEventDuration(e=> TimeSpan.MaxValue)
    		.ClipEventDuration(str2, (e1, e2) => e1.ItemId == e2.ItemId)
    	select e;
    
    var resultStream = from l in referenceStream
    		.ShiftEventTime(e => TimeSpan.FromTicks(1))
    	join r in str2
    	on l.ItemId equals r.ItemId
    	select r;

    I added the ItemId field in the code sample because that seems to be the pattern we have been using around here. If you don't have a field that serves as the unique identifier for an event, you just have to change your code to something like this:

    var referenceStream = from e in str1
    		.AlterEventDuration(e=> TimeSpan.MaxValue)
    		.ClipEventDuration(str2, (e1, e2) => true)
    	select e;
    
    var resultStream = from l in referenceStream
    		.ShiftEventTime(e => TimeSpan.FromTicks(1))
    	from r in str2
    	select r;


    • Düzenleyen TXPower125 02 Mayıs 2012 Çarşamba 15:04 Added another example.
    • Yanıt Olarak Öneren TXPower125 02 Mayıs 2012 Çarşamba 15:04
    • Yanıt Olarak İşaretleyen Jayanta Dey 03 Mayıs 2012 Perşembe 05:39
    02 Mayıs 2012 Çarşamba 13:54

Tüm Yanıtlar

  • Here's how I got this working in LinqPad.

    1. Extend and clip the reference stream to the data stream. This will convert the reference stream to a signal whose events start at the original event start time while the end time is set to just before the start time of a matching event in the data stream.
    2. Now shift the event time of the events in the reference stream by 1 tick and join with the data stream. This will cause the events in the reference stream to overlap with the events in the data stream.

    Here's the code:

    var referenceStream = from e in str1
    		.AlterEventDuration(e=> TimeSpan.MaxValue)
    		.ClipEventDuration(str2, (e1, e2) => e1.ItemId == e2.ItemId)
    	select e;
    
    var resultStream = from l in referenceStream
    		.ShiftEventTime(e => TimeSpan.FromTicks(1))
    	join r in str2
    	on l.ItemId equals r.ItemId
    	select r;

    I added the ItemId field in the code sample because that seems to be the pattern we have been using around here. If you don't have a field that serves as the unique identifier for an event, you just have to change your code to something like this:

    var referenceStream = from e in str1
    		.AlterEventDuration(e=> TimeSpan.MaxValue)
    		.ClipEventDuration(str2, (e1, e2) => true)
    	select e;
    
    var resultStream = from l in referenceStream
    		.ShiftEventTime(e => TimeSpan.FromTicks(1))
    	from r in str2
    	select r;


    • Düzenleyen TXPower125 02 Mayıs 2012 Çarşamba 15:04 Added another example.
    • Yanıt Olarak Öneren TXPower125 02 Mayıs 2012 Çarşamba 15:04
    • Yanıt Olarak İşaretleyen Jayanta Dey 03 Mayıs 2012 Perşembe 05:39
    02 Mayıs 2012 Çarşamba 13:54
  • To make life a little easier, you can also use the FoldPairs macro that is in the LinqPad samples for StreamInsight. It's under "What's new in StreamInsight 1.2".

    DevBiker (aka J Sawyer)
    Microsoft MVP - Sql Server (StreamInsight)


    Ruminations of J.net


    If I answered your question, please mark as answer.
    If my post was helpful, please mark as helpful.

    02 Mayıs 2012 Çarşamba 19:01
    Moderatör
  • The FoldPairs macro in LinqPad, as is, only works on 1 stream. If you have 2 seperate streams of data, like in the original post, you'll have to modify the FoldPairs macro to take in both streams.

    02 Mayıs 2012 Çarşamba 19:08
  • Thanks, your solution works perfectly.
    03 Mayıs 2012 Perşembe 05:46