locked
Analyzing Azure Storage Queue data in “real-time” RRS feed

  • Question

  • I have access to a Azure Storage Queue where I receive a message with sales data (integer, 4B pr. message) each minute. I want to analyze this data in near real-time in some frontend visualization tool, in this case Power BI. I want to persist the last 24 hours of data for real-time streaming, and then store the older-than-24-hours data somewhere for later use.

    My issue is, I have limited experience with Azure, so I was wondering what would be a cost-efficient and scalable solution for this problem in Azure (I have created a test account on the service)?

    Thanks in advance. 

    Wednesday, May 27, 2020 1:31 PM

All replies

  • Real time processing deals with streams of data that are captured in real-time and processed with minimal latency to generate real-time (or near-real-time) reports or automated responses. For example, a real-time traffic monitoring solution might use sensor data to detect high traffic volumes. This data could be used to dynamically update a map to show congestion, or automatically initiate high-occupancy lanes or other traffic management systems.
    Wednesday, May 27, 2020 2:00 PM
  • Yes, I agree with what you write, but I don't see how it answers my question? :-) 
    Wednesday, May 27, 2020 2:13 PM
  • Apologies for the delay in responding here! It depends what exactly do want to measure. 

    You can use the Azure Monitor in this scenario and let me know if that helps!  Azure Storage monitoring data reference

    Azure Storage Analytics performs logging and provides metrics data for a storage account. You can use this data to trace requests, analyze usage trends, and diagnose issues with your storage account.

    https://docs.microsoft.com/en-us/azure/storage/common/storage-analytics

    Hope this helps! 

    Kindly let us know if the above helps or you need further assistance on this issue.
    ------------------------------------------------------------------------------------------

    Do click on "Mark as Answer" and Upvote on the post that helps you, this can be beneficial to other community members.

    Friday, June 5, 2020 12:10 PM
  • Is there any update on the issue?

    If the suggested answer helped for your issue, do click on "Mark as Answer" and “Vote as Helpful” on the post that helps you, this can be beneficial to other community members.

    Monday, June 8, 2020 10:47 AM