Downsampling RRS feed

  • Question

  • Hi

    I trying to create a downsampling ASA job, but I have a hard time getting it to work. Every 10 minute a package of 600 data points (1 pr. sec) are being stored in blobstore, I need to split these in windows of 1 minute, based on datatime,  and calculate average. I've tried using both tumblingwindow(second,60) and sessionwindow(second,2,60), but I seem to get mixed results, sometimes I just get one output with average of 60 points, sometime I get one output with with average of 600, and sometimes it seems to work correctly... I can't seem to find a solution that works everytime....

    My current ASA look like this:

    SELECT DataPointAlias.ArrayValue.DataPointName AS DataPointName, Count(*) AS RecordCount, AVG(DataPointAlias.ArrayValue.DataValue) AS DataValue, MAX(CAST(HfData.CreateTime AS DateTime)) AS WindowEndTime INTO DownsampleOutTable FROM HfInput AS HfData TIMESTAMP BY CAST(HfData.CreateTime AS DateTime) CROSS APPLY GetArrayElements(HfData.DataPoints) AS [DataPointAlias] GROUP BY sessionwindow(second, 2, 60), DataPointAlias.ArrayValue.DataPointName

    And my input data are on the form:

        "DataPoints": [{
            "DataPointName": "RPM",
            "DataValue": "100",
        "CreateTime": "2018-07-05T09:21:13+00:00",
    }, {
        "DataPoints": [{
            "DataPointName": "PRM",
            "DataValue": "101",
        "CreateTime": "2018-07-05T09:21:14+00:00"
    ... 600 datapoints, one every second

    Any help would be greatly appreciated, TIA


    • Edited by smolesen Thursday, July 12, 2018 5:30 AM
    Thursday, July 12, 2018 5:26 AM

All replies

  • As you say that only at sometimes it seems to work correctly. Can you please give me a sample dataset for which it fails to execute? 
    Monday, July 23, 2018 11:29 PM