locked
Unexpected "Job will exceed the maximum amount of Event Hub Receivers" error when starting Stream Analytics job. RRS feed

  • Question

  • I have a simple three-part Event Hub -> Stream Analytics -> SQL Server pipeline in Azure.

    • I have one event hub, which has one consumer group.
    • The event hub is read exclusively by one Stream Analytics job.
    • The stream analytics job has five different SQL table outputs.
    • The stream analytics query consists of five consecutive statements of the form "SELECT Id, Data INTO [table1] FROM [MyEventHub] WHERE TableTarget = 1" for 1 through 5.

    Using one input (the Event Hub) and five outputs, the job runs fine.

    I can add a sixth, unused output to the Stream Analytics job, and the job still runs fine.

    However, when I add the corresponding "SELECT Id, Data INTO [table6] FROM [MyEventHub] WHERE TableTarget = 6" statement, the Stream Analytics job fails to start with a message "The streaming job failed: Stream Analytics job has validation errors: Job will exceed the maximum amount of Event Hub Receivers." 

    As far as I can tell, there is one Event Hub Receiver (the Stream Analytics job). I have two questions:

    • How is the number of "Event Hub Receivers" calculated for Stream Analytics jobs like this?
    • Do I need to make a new Stream Analytics job (and consumer group on the Event Hub) for every five outputs?


    • Edited by Emdel Monday, September 19, 2016 7:36 PM
    Monday, September 19, 2016 7:27 PM

Answers

  • Hi Emdel,

    You have more than 5 select statement in your one ASA job connected to same input (pointing to a consumer group).

    HTH


    Sudhir Rawat


    Sunday, September 25, 2016 1:23 PM

All replies

  • Hi Emdel,

    You were hitting limit from service bus side. One consumer group can have maximum 5 readers.  

    • How is the number of "Event Hub Receivers" calculated for Stream Analytics jobs like this? [Sudhir] Mentioned above
    • Do I need to make a new Stream Analytics job (and consumer group on the Event Hub) for every five outputs? [Sudhir] Actually not. What you can do is create another input connecting to same input but different consumer group and use this input from 6th query onwards till 5 readers.

    Hope it helps.


    Sudhir Rawat

    Tuesday, September 20, 2016 6:12 AM
  • I'm still not sure I understand why the Service Bus is telling me that it has five readers; as far as I can tell, my consumer group has one reader (a single Stream Analytics job). 
    Friday, September 23, 2016 4:22 PM
  • Hi Emdel,

    You have more than 5 select statement in your one ASA job connected to same input (pointing to a consumer group).

    HTH


    Sudhir Rawat


    Sunday, September 25, 2016 1:23 PM