locked
Error Extending Event Duration RRS feed

  • Question

  • Hi, I've got a query like so, where my goal was to extend the duration so that I'd see the sum of events that stretch across a 60 minute time window.  I followed the example from the Help Docs for stretching the duration.

    var

     

    slidingSum = from window in inputDataStream.AlterEventDuration(e => e.EndTime.Subtract(e.StartTime).Add(TimeSpan.FromMinutes(60))).Snapshot()

     

    select new { sum = window.Sum(e => e.OrderQuantity) };

    However when I execute the following query, I get an error.

    Query

     

    query = slidingSum.ToQuery(typeof(TextFileOutputFactory), outputConf, EventShape.Point, StreamEventOrder.FullyOrdered);

     

    The error is:
    [Microsoft.ComplexEventProcessing.Linq.QueryGenerationException] {"Only static user-defined function call is supported. The following method call is not supported: 'e.EndTime.Subtract(e.StartTime).Add(FromMinutes(60))'."}

    Can I not do this on a point event?  Or is there something else I'm missing here?

    Thursday, February 4, 2010 4:22 PM

Answers

  • To add grouping, modify the query like this.

    var

     

     

     

    myquery = from input in inputDataStream 
                        group input by new {Product = e.Product} into groupingStream 
                        from w in groupingStream.HoppingWindow(
    TimeSpan.FromSeconds(30), TimeSpan
    .FromMilliseconds(100))

     

    select new { Sum = w.Sum(e => e.OrderQuantity)};

    Let me know if that works for you.

    I am curious to find out why the alterlifetime query does not work. It does work for me (but I have the latest version on my box and not the CTP3 version).


    -------------------------------------------------------------------------------- Asvin Ananthanarayan MS StreamInsight Team Disclaimer: This posting is provided "AS IS" with no warranties, and confers no rights.
    Friday, February 5, 2010 9:57 PM

All replies

  • Actually, got this working by using a Hopping Window to define a broad window with small intervals.

    var

     

    myquery = from w in inputDataStream.HoppingWindow(TimeSpan.FromSeconds(30), TimeSpan.FromMilliseconds(10))

     

    select new { Sum = w.Sum(e => e.OrderQuantity)};

    Thursday, February 4, 2010 11:38 PM
  • Ok, I lied.  I still want to extend the duration so that performing an aggregation of all events that occured in the past x time catches all events that came through at that time.  Think I can only do that by extending the point event end time.

    So, any thoughts on the original error?
    Friday, February 5, 2010 5:12 AM
  • this should work. The problem here could be the anonymous type that gets created because of

    select new { Sum = w.Sum(e => e.OrderQuantity)};

    Could you create an explicit type like

    class StreamOutputType
    {
          public Sum {get; set;}
    }

    and then rewrite the select portion of the query as

    select new StreamOutputType{ Sum = w.Sum(e => e.OrderQuantity)};

    If this works then the problem is a bad error message which is fixed in the final release of the product.

    -------------------------------------------------------------------------------- Asvin Ananthanarayan MS StreamInsight Team Disclaimer: This posting is provided "AS IS" with no warranties, and confers no rights.
    Friday, February 5, 2010 6:54 PM
  • Thanks for the suggestion, but defining an explicit type still resulted in the same error.  Whether this is a bad error message or not, I'd still love advice for taking in point events, and putting them into a window of a particular length so that I can aggregate them.

    This query (like shown above) kind of does the trick, but then I can't seem to add grouping to this:

    var

     

    myquery = from w in inputDataStream.HoppingWindow(TimeSpan.FromSeconds(30), TimeSpan.FromMilliseconds(100))

     

    select new { Sum = w.Sum(e => e.OrderQuantity)};

    This results in 30 second windows (and unless I make the interval tiny, nothing shows up) but I'm not sure how to add a valid grouping statement to this so that I could, say, group based on e.Product and show the sum of orders per product identifier.

    Thoughts?

    Friday, February 5, 2010 8:47 PM
  • To add grouping, modify the query like this.

    var

     

     

     

    myquery = from input in inputDataStream 
                        group input by new {Product = e.Product} into groupingStream 
                        from w in groupingStream.HoppingWindow(
    TimeSpan.FromSeconds(30), TimeSpan
    .FromMilliseconds(100))

     

    select new { Sum = w.Sum(e => e.OrderQuantity)};

    Let me know if that works for you.

    I am curious to find out why the alterlifetime query does not work. It does work for me (but I have the latest version on my box and not the CTP3 version).


    -------------------------------------------------------------------------------- Asvin Ananthanarayan MS StreamInsight Team Disclaimer: This posting is provided "AS IS" with no warranties, and confers no rights.
    Friday, February 5, 2010 9:57 PM
  • I'm curious too ;)

    Thanks for the grouping tip ... worked great.  It would be nice to actually be able to extend the duration so that I don't have to take such small intervals in my hopping window in order to get results.

    I'll look forward to the next code drop to see if this fixes my error.
    Friday, February 5, 2010 10:21 PM
  • you should also be able to do larger hopping windows with spans of 60 minutes. You might not be seeing output because we have to hold back CTIs to the start of the window for hopping windows. Try sending a CTI with DateTime.MaxValue at the end of your stream as an experiment. It should work and then you can play around with the the CTI frequency
    -------------------------------------------------------------------------------- Asvin Ananthanarayan MS StreamInsight Team Disclaimer: This posting is provided "AS IS" with no warranties, and confers no rights.
    Friday, February 5, 2010 10:42 PM