How to make the output as another input? RRS feed

  • Question

  • The case: We have a 1day aggregation window which sum the total value received from Event hub (1, 2, 3 … each minute send a value), we set the output to a blob called 1dayresult. Now we want to get the blob data as another 1week aggregation input, each week we want to get the data from the blob and do calculation, so can we set the 1day result blob as the input for the 1week aggregation? We know we can set the window unit is 7day, but we think it will slow down the performance, because if we make the 1day result blob as the input, we only need 7 values, but if we use the 7day window, we will get more than 7*24*60 values and then do the calculation. We also want to have a month aggregation, but the The maximum size of the window is 7 days. So how to achieve it? 

    Wednesday, May 18, 2016 9:58 AM


  • I suggest you use following solution

    1. Create ASA job with 1 day aggregation and use blob output

    2. Use spark streaming to do 7 days aggregation.

    3. Create another ASA job that uses blob input (the previous generated blob path) and do 7 days aggregation.

    2 or 3 for 7 days aggregation,

    Wednesday, May 25, 2016 4:54 PM