locked
Receiving events from event hub was blocked. RRS feed

  • Question

  • In our Cloud Service project, we have 2 instances for work role (deploy to Azure), the work role is consume events from the EventHub using EventProcessorHost(host name is RoleInstance name).

    For sending events:

        var client = EventHubClient.CreateFromConnectionString(serviceBusConnectionString, hubName);

                while (true)

                {

                    var eventData = new EventData(Encoding.UTF8.GetBytes("test")) {PartitionKey = "key"};

                        eventData.Properties.Add("time", DateTime.UtcNow);

                   

                    client.SendAsync(eventData).Wait();

                   

                    Thread.Sleep(50);

                }

    Each 50ms, we send one event (event1, 2,3 …….);

    For receiving data:    

     public async Task ProcessEventsAsync(PartitionContext context, IEnumerable<EventData> events)

            {

                //when we get the event, so we can view the log

                Trace.WriteLine(“got events”);

                foreach (var eventData in events)

                {

                    // handle the event

                    Task.Delay(12000).Wait();

                   

                }

                await ContextCheckpointAsync(context);

            }

    We add the delay for event operation.

    It seems that we cannot receive data in time from the log, seems event6 was blocked for the Event5 delay, after the 12ms, we can receive event6 from the EventHub, and the Event6 delay is 40s(from the log, we send event6 to Hub at 35:10, but we get from Hub at 35:50),

    So I wonder to know the maximum number of threads are working on processing fot the EventProcessorHost? Depends on the Partitions?

    And is there any way to receiving events in time?

    Thursday, March 5, 2015 11:42 AM

Answers

  • Hi Jordan

    Since Task.Delay call blocks the callback, host won't hand over new events until you're done with the current batch. This is due to order guarantee of the events delivered, i.e. host should process the events in order from the same partition.

    If event process is taking so long then you should consider to move process job into a separate thread so host can deliver new batch of events while thread is working on the previous batch.

    • Marked as answer by Jordan--23 Friday, March 6, 2015 2:00 AM
    Thursday, March 5, 2015 6:17 PM