none
PREEMPTIVE_OS_GETPROCADDRESS

    Question

  • We have a 64-bit Enterprise server w/Svc pack 1.  Running SS 2008 64 bit (of course).  2 four-way CPU's.  56 GB of RAM. 

    In the last month, it has froze 3 times at 11 am;  1PM & 2AM.  Each time the Log backup was seen to have entered a never-stop condition.  Normally the log backups run in 6 seconds.  Analysing wait states shows that PREEMPTIVE_OS_GETPROCADDRESS is the by-far highest waitstate.  I have been unable to find any documentation on PREEMPTIVE_OS_GETPROCADDRESS.

    When not 'frozen' this sql machine has no issues.  There is one other OS issue i should mention:  three minutes before the last event (as identified by the log backup going to 6 hour duration) the os reported "VOLSNAP"...  "The description for Event ID '-1073348583' in Source 'VolSnap' cannot be found".  Could there be a connection there?


    steve
    Tuesday, July 26, 2011 7:47 PM

All replies

  • Hi steve,

    Thank you for your question.
     
    I am currently looking into this issue and will give you an update as soon as possible.
     
    Thank you for your understanding and support.


    Best Regards,
    Stephanie Lv

    Forum Support
    Please remember to mark the replies as answers if they help and unmark them if they provide no help. If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    Thursday, July 28, 2011 6:49 AM
  • Hi Steve,

     

    The VolSnap thing is part of the VSS service as far as I know, could this be causing I/O freezing on the drive in the same way NTBackup used to do if you tried to backup a drive with SQL files on it? Have you any non SQL backups enabled on the server or VSS set up? I'm purely guessing here by the way, this may in no way be the problem.

     

    Thanks,

     

    Stephen

    Thursday, July 28, 2011 9:55 AM
  • Hey Steve,

     

    Without the SQL Server errorlogs, the OS eventlogs and potentially capturing an incident with a tool called PPSDiag, we are making some assumptions and limited on suggestions. 

    First, we are assuming that you are using VDI backups instead of SQL Server backups?  VDI = 3rd party or hardware level type of backups that will cause SQL Server to freeze and thaw IO for doing backups.  http://blogs.msdn.com/b/psssql/archive/2009/03/03/how-it-works-sql-server-vdi-vss-backup-resources.aspx

    Since the problem happens at a specific time, you could investigate to see what other kinds of system related operations could be going on that could be affecting this behavior.  A common cause of this type of issue is the expiration of log backups by the 3rd party backup that is taking a long time to clear out.  Possibly the reason it only happens at that time is that is when it cleans up expired log backups? Just a thought.

    The waittype PREEMPTIVE_OS_GETPROCADDRESS can include the execution time of calling into a extended stored procedure (XPROC).  A lot of third party backups are implemented through XPROCs.  So, the delay could be in the behavior of the XPROC.

    Running performance monitor during the problem time could help to see what is going on with the system as a whole and in particular the disks.

    Examine the settings used in your 3rd party backup to see if you have set it up to do something special at 11:00 AM.

    You do not mention if you are using Windows 2008 or Windows 2008 R2? 

    We do recommend that you are running the latest versions of the OS and build for the version of SQL Server you are using.  If the problem still occurs after this, we will need to collect some data.  Calling into Support for assistance is the best means of getting this level of detailed analysis.

    Sincerely,

    Rob Beene, MSFT

    Thursday, July 28, 2011 7:03 PM
  • We're running Windows 2008  Version 6.0 (build 6001: Service Pack 1)

    We're running SQL 2008 RTM.

    We're running Native SQL 2008 backups (no compression).

    We're currently focussed on Diskkeeper as a potential source of the issue.

    Our underlying disk structure is a single set of 12 300 SAS GB drives raid-5 together to form the E drive where all our data (and logs) is.  (No, i didn't design this.) 

    I have recently limited the SQL to max memory of 48 GB. I had hoped that that would solve the issue, but it froze again this morning @3:34 AM.

     

    I have proposed upgrading the OS and the SQL to  R2 versions and am waiting on a response. 

    Thank you for your time and comments.

    Steve


    steve
    Friday, July 29, 2011 4:20 PM
  • Hey Steve,

    Thank you for the additional information.  You do want to make sure you are also running the latest version of Diskkeeper as well.  The error you previously posted does seem specific to something outside of SQL Server at this point such as Diskkeeper.  Using Performance Monitor can also help to see how the disks are performing.  The "freeze" or "hang" could be a result of disk performance.  Of course this is an assumption or guess without having any actual data to confirm.

     

    You could look in SQL Server's errorlog to see if there are any kind of the following error messages which could (but not necessarily) indicate slow disk performance as a possibility.

    1. Time out occurred while waiting for buffer latch

    OR

    2. I/O requests taking longer than 15 seconds to complete

     

    Although being on 2008 R2 is a good idea, I actually meant make sure you are on the latest build for SQL Server 2008.  Currently, the latest build is SQL Server 2008 Service Pack 2 with Cumulative Update 5.

    968382 How to obtain the latest service pack for SQL Server 2008
    http://support.microsoft.com/default.aspx?scid=kb;EN-US;968382

     2402659 The SQL Server 2008 builds that were released after SQL Server 2008 Service Pack 2 was released
    http://support.microsoft.com/default.aspx?scid=kb;EN-US;2402659

    You would want to regularly check the last link as every 2 months we release a cumulative Update.  Also, we always recommend testing a Service Pack or Cumulative Update in a test environment before applying to production to limit the possibility of an issue related to a change made in a Service Pack or Cumulative update.

    I hope this information helps.

     

    Sincerely,

     

    Rob Beene, MSFT

    Monday, August 01, 2011 12:07 PM
  • could you draw a time pattern of all occurrences? happens randomly?

    when it freezes, how would you know that? Could you still get into system?

    Maybe get a manual memory dump of MSSQL to send to CSS to analysis.


    If you think my suggestion is useful, please rate it as helpful.
    If it has helped you to resolve the problem, please Mark it as Answer.
    http://twitter.com/7Kn1ghts

    Monday, August 01, 2011 12:34 PM
  • Can you check if the backup job is actually doing work?

    As Robert pointed out, the wait type could be misleading at times. You would need to check if the cpu time, read/writes for the session id showing the wait type is showing an increase in value(s) with the status being runnable or running. If that is the case, then you have a resource bottlneck on your system like disk I/O.

    Reference: http://troubleshootingsql.com/2011/07/20/preemptive_os_getprocaddress-are-you-confusing-me/


    This posting is provided "AS IS" with no warranties, and confers no rights.
    My Blog: http://www.troubleshootingsql.com
    Twitter: @banerjeeamit
    MSDN Blog: http://blogs.msdn.com/sqlserverfaq
    Tuesday, August 02, 2011 6:58 PM
  • Amit, SevenKnights, Robert,

      thank you all for your replies.  I am currently waiting on the machine owner to get back to me on several questions.  Otherwise i would have replied sooner. 

    Amit...The backup is a log backup and it is stalled.

    SevenKnights the timing is clearly random.  I'm waiting to get back info from the operators to know whether they can get onto the box during the "freeze" time.


    steve
    Wednesday, August 03, 2011 7:34 PM
  • Do you see the IO being done by the session increasing during that time when you look at multiple snapshots of sys.dm_exec_requests? And are there any errors pertaining to VSS or SQLWriter in the SQL Server Errorlogs or Application logs during that time?
    This posting is provided "AS IS" with no warranties, and confers no rights.
    My Blog: http://www.troubleshootingsql.com
    Twitter: @banerjeeamit
    MSDN Blog: http://blogs.msdn.com/sqlserverfaq
    Thursday, August 04, 2011 4:25 PM
  • I am facing the same problem on SQLServer 2008 R2. ( Microsoft SQL Server 2008 R2 (SP1)). My backup has got stalled 3 times in the last 7 days. surprising thing is that I have another server with the same version and that backup runs absolutely fine. Even the backup destination is same. only difference is sometimes my full backup gets stalled. any help will be appreciated.

    Robert

    not seeing

     any Time out occurred while waiting for buffer latch neither am I seeing an I/O requests taking longer than 15 seconds to complete

    Mak

    Wednesday, August 22, 2012 9:27 AM