none
Azure Stream Analytics not outputting all input events RRS feed

  • Question

  • We have a system that sends tag data through Open Automation Software to an Azure IOT HUB instance and then on to Azure Stream Analytics that then outputs to an Azure SQL DW.

    The system works correctly (ie – SA outputs all input events) up to 26 tags worth of data (data events are small, .16 KB) every 30 seconds.

    At 27 tags worth of data SA begins to fail on outputs from the very beginning of the test run. All of the input events from IOT HUB are received, however only a subset of those events are output by SA to the SQL DW. SU % Utilization is only 10, so there is plenty of memory available. From investigation then there’s apparently no ability to partition the SA output as the target is one that is automatically managed by Azure.

    The SA job diagram does show input events backlogged upon occasion, but the delta between that count and the number of output events is nowhere near the same.  The Activity Log shows no Write or other errors.

    What are we missing that could correct this and allow the platform to scale correctly?



    • Edited by bluesurf Thursday, September 6, 2018 2:22 AM
    Wednesday, September 5, 2018 8:22 PM

All replies

  • I have verified with our engineers, from the information provided by you this doesn't seem like ASA issue. The issue might be due to the query logic or SQL destination schema
    Wednesday, September 12, 2018 8:22 PM
    Moderator