Notification Services: At least one performance monitoring interval was skipped

Question

Hi All,

Recently one of the Notification Service instances we are responsible for has begun reporting the following error in the event log.  This occurs about once every 10 seconds.

• At least one performance monitoring interval was skipped.

 

All of our research into this issue points to corrupted performance counters, so we have already tried the instructions here: http://support.microsoft.com/kb/827963

 

Unfortunitely this has not resolved the problem for us, so we are looking for extra information:

 

1. Does anyone know what the actual effect on the system will be with this event being logged?  Is it safe to assume (for the short term) that this just affects performance counters being logged, rather than notifications themselves?

 

2. Has anyone had any success with other methods to resolve this issue?

 

Thanks in advance for any suggestions or advice you can offer.

 

Regards,

 

Nick Hoggard

 

Full Allert Details:

-------------------------   Warining.  Source:NotificationServices      Event ID:1550  -------------------------------------------------

At least one performance monitoring interval was skipped.

EventParameters:

  Application Name: {Application Name Removed}

  Count of Skipped Performance Monitor Intervals: 1

 

InstanceName: {Instance Name Removed}

ApplicationName: {Application Name Removed}

Component: Performance Monitor

Thread: 9

 

---------------------------  Error. Source: NotificationServices            Event ID:5501  --------------------------------------------------

Description: The performance monitor failed to get event counter values to update the performance counters.

EventParameters:

 InstanceName: {Instance Name Removed}

 ApplicationName: {Application Name Removed}

  Event Class Name: {Event Class Name Removed}

 

SqlServerError:

  Source: .Net SqlClient Data Provider

  Number: -2

  State: 0

  Class: 11

  Server: {Server / Instance Names Removed}

  Message: Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.

  Procedure:

  Line Number: 0

 

Description: Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.

InstanceName: {Instance Name Removed}

ApplicationName: {Application Name Removed}

Component: Performance Monitor

Thread: 9

 

Answer 1

Hi Nick,

It does not sound like you are losing notifications; but this is not perfmon counter corruption either.

Event ID:1550

This error means that we committed to update the perfmon counter, let's say every x seconds, but failed to do so. SQL Notification Services (NS) calculates the amount of time in milli-seconds for the thread that updates perfmon counters to sleep until the next update. When the timer expires, a function call to increment the perfmon counter is then started. After the perfmon counter update completes, more time has passed than expected, perfmon may have been updated or not, but it took more time than expected so we basically lost a chance to update again as the clock hand has moved. We calculate a new sleep interval for the next perfmon counter update by increasing the interval time.

Event ID:5501

SQL NS queries the database for perfmon counter values. SQL NS executes a query to collect the information from the SQL instance database (AwaitingEventBatches, EventBatchesSubmitted, TotalEvents, for example). This query times out.

I would say that both events are related and are caused by a query that times out when trying to collect perfmon data for the SQL NS instance. This code calls a stored procedure named [NS????PerfCounters].  You can try to investigate why it is timing out from a performance perspective, or simply ignore the error.

Timeouts are indicators that this SQL NS instance database may be suffering some performance degredation caused by database size or machine load.

Thanks,

Cathy Miller

Microsoft Online Community Support

Answer 2

Hi Cathy,

 

Thanks for the details provided.  This certainly does provide some more information for us to work from - I'll get the team to have another look at this now that we have this context.

 

Cheers,

 

Nick

 

Answer 3

There is a program that can help to solve this issue, here is the link http://www.real-user-monitoring.com