none
SQL Server Stack DUMP Generated

    Question

  • We just had our SQL Server crash last week with a Crash dump generated, a bit about the Environment (This is a multi instance Environment)

    Server OS: Windows 2008 R2 Enterprise,64 bit
    Memory: 256 GB
    SQL Server: 2008 R2 Enterprise Edition
    Max Server Memory: 60 GB
    Processor Count:  16

    I am attaching Excerpts from the SQL Servr Error log (Below) while SQL Server went down. Can someone help me analyze this.

    2011-10-07 10:45:49.07 Server      Error: 17311, Severity: 16, State: 1.
    2011-10-07 10:45:49.07 Server      SQL Server is terminating because of fatal exception e06d7363. This error may be caused by an unhandled Win32 or C++ exception, or by an access violation encountered during exception handling. Check the SQL error log for any related stack dumps or messages. This exception forces SQL Server to shutdown. To recover from this error, restart the server (unless SQLAgent is configured to auto restart).
    2011-10-07 10:45:49.14 Server      Using 'dbghelp.dll' version '4.0.5'
    2011-10-07 10:45:49.16 Server      **Dump thread - spid = 0, EC = 0x0000000000000000
    2011-10-07 10:45:49.16 Server      ***Stack Dump being sent to C:\Program Files\Microsoft SQL Server\MSSQL10_50.DWDEV1\MSSQL\LOG\SQLDump0001.txt
    2011-10-07 10:45:49.16 Server      * *******************************************************************************
    2011-10-07 10:45:49.16 Server      *
    2011-10-07 10:45:49.16 Server      * BEGIN STACK DUMP:
    2011-10-07 10:45:49.16 Server      *   10/07/11 10:45:49 spid 24308
    2011-10-07 10:45:49.16 Server      *
    2011-10-07 10:45:49.16 Server      * ex_handle_except encountered exception E06D7363 - Server terminating
    2011-10-07 10:45:49.16 Server      *
    2011-10-07 10:45:49.16 Server      *
    2011-10-07 10:45:49.16 Server      *  MODULE                          BASE      END       SIZE
    2011-10-07 10:45:49.16 Server      * sqlservr                       0000000000B00000  00000000046ADFFF  03bae000
    2011-10-07 10:45:49.16 Server      * ntdll                          0000000077880000  0000000077A2BFFF  001ac000
    2011-10-07 10:45:49.16 Server      * kernel32                       0000000077660000  000000007777EFFF  0011f000
    2011-10-07 10:45:49.16 Server      * KERNELBASE                     000007FEFDA00000  000007FEFDA6AFFF  0006b000
    2011-10-07 10:45:49.16 Server      * ADVAPI32                       000007FEFE100000  000007FEFE1DAFFF  000db000
    2011-10-07 10:45:49.16 Server      * msvcrt                         000007FEFF520000  000007FEFF5BEFFF  0009f000
    2011-10-07 10:45:49.16 Server      * sechost                        000007FEFF5C0000  000007FEFF5DEFFF  0001f000
    2011-10-07 10:45:49.16 Server      * RPCRT4                         000007FEFF760000  000007FEFF88DFFF  0012e000
    2011-10-07 10:45:49.16 Server      * MSVCR80                        00000000753D0000  0000000075498FFF  000c9000
    2011-10-07 10:45:49.16 Server      * MSVCP80                        00000000754A0000  00000000755A8FFF  00109000
    2011-10-07 10:45:49.16 Server      * sqlos                          0000000071060000  0000000071066FFF  00007000
    2011-10-07 10:45:49.16 Server      * Secur32                        000007FEFD3F0000  000007FEFD3FAFFF  0000b000
    2011-10-07 10:45:49.16 Server      * SSPICLI                        000007FEFD680000  000007FEFD6A4FFF  00025000
    2011-10-07 10:45:49.16 Server      * pdh                            000007FEFA920000  000007FEFA96DFFF  0004e000
    2011-10-07 10:45:49.16 Server      * SHLWAPI                        000007FEFF930000  000007FEFF9A0FFF  00071000
    2011-10-07 10:45:49.16 Server      * GDI32                          000007FEFF250000  000007FEFF2B6FFF  00067000
    2011-10-07 10:45:49.16 Server      * USER32                         0000000077780000  0000000077879FFF  000fa000
    2011-10-07 10:45:49.16 Server      * LPK                            000007FEFDDC0000  000007FEFDDCDFFF  0000e000
    2011-10-07 10:45:49.16 Server      * USP10                          000007FEFDBB0000  000007FEFDC79FFF  000ca000
    2011-10-07 10:45:49.16 Server      * USERENV                        000007FEFCB00000  000007FEFCB1DFFF  0001e000
    2011-10-07 10:45:49.16 Server      * profapi                        000007FEFD7E0000  000007FEFD7EEFFF  0000f000
    2011-10-07 10:45:49.16 Server      * WINMM                          000007FEF6A80000  000007FEF6ABAFFF  0003b000
    2011-10-07 10:45:49.16 Server      * IPHLPAPI                       000007FEFBA00000  000007FEFBA26FFF  00027000
    2011-10-07 10:45:49.16 Server      * NSI                            000007FEFDDB0000  000007FEFDDB7FFF  00008000
    2011-10-07 10:45:49.16 Server      * WINNSI                         000007FEFB9F0000  000007FEFB9FAFFF  0000b000
    2011-10-07 10:45:49.16 Server      * opends60                       0000000071050000  0000000071057FFF  00008000
    2011-10-07 10:45:49.16 Server      * NETAPI32                       000007FEFB780000  000007FEFB795FFF  00016000
    2011-10-07 10:45:49.16 Server      * netutils                       000007FEFCD60000  000007FEFCD6BFFF  0000c000
    2011-10-07 10:45:49.16 Server      * srvcli                         000007FEFD330000  000007FEFD352FFF  00023000
    2011-10-07 10:45:49.16 Server      * wkscli                         000007FEFB750000  000007FEFB764FFF  00015000
    2011-10-07 10:45:49.16 Server      * LOGONCLI                       000007FEFCF20000  000007FEFCF4FFFF  00030000
    2011-10-07 10:45:49.16 Server      * SAMCLI                         000007FEFA560000  000007FEFA573FFF  00014000
    2011-10-07 10:45:49.16 Server      * BatchParser                    0000000071020000  000000007104CFFF  0002d000
    2011-10-07 10:45:49.16 Server      * IMM32                          000007FEFE2F0000  000007FEFE31DFFF  0002e000
    2011-10-07 10:45:49.16 Server      * MSCTF                          000007FEFE1E0000  000007FEFE2E8FFF  00109000
    2011-10-07 10:45:49.16 Server      * psapi                          0000000077A40000  0000000077A46FFF  00007000
    2011-10-07 10:45:49.16 Server      * instapi10                      0000000074B70000  0000000074B7CFFF  0000d000
    2011-10-07 10:45:49.16 Server      * cscapi                         000007FEF6A70000  000007FEF6A7EFFF  0000f000
    2011-10-07 10:45:49.16 Server      * sqlevn70                       0000000070E10000  0000000071010FFF  00201000
    2011-10-07 10:45:49.16 Server      * CRYPTSP                        000007FEFD060000  000007FEFD076FFF  00017000
    2011-10-07 10:45:49.16 Server      * rsaenh                         000007FEFCD70000  000007FEFCDB6FFF  00047000
    2011-10-07 10:45:49.16 Server      * CRYPTBASE                      000007FEFD710000  000007FEFD71EFFF  0000f000
    2011-10-07 10:45:49.16 Server      * BROWCLI                        000007FEF68C0000  000007FEF68D1FFF  00012000
    2011-10-07 10:45:49.16 Server      * AUTHZ                          000007FEFD190000  000007FEFD1BEFFF  0002f000
    2011-10-07 10:45:49.16 Server      * MSCOREE                        000007FEFA360000  000007FEFA3CEFFF  0006f000
    2011-10-07 10:45:49.16 Server      * ole32                          000007FEFF310000  000007FEFF511FFF  00202000
    2011-10-07 10:45:49.16 Server      * credssp                        000007FEFCBB0000  000007FEFCBB8FFF  00009000
    2011-10-07 10:45:49.16 Server      * msv1_0                         000007FEFCE80000  000007FEFCED0FFF  00051000
    2011-10-07 10:45:49.16 Server      * cryptdll                       000007FEFD360000  000007FEFD373FFF  00014000
    2011-10-07 10:45:49.16 Server      * kerberos                       000007FEFD090000  000007FEFD142FFF  000b3000
    2011-10-07 10:45:49.16 Server      * MSASN1                         000007FEFD880000  000007FEFD88EFFF  0000f000
    2011-10-07 10:45:49.16 Server      * schannel                       000007FEFCDC0000  000007FEFCE16FFF  00057000
    2011-10-07 10:45:49.16 Server      * CRYPT32                        000007FEFD890000  000007FEFD9F5FFF  00166000
    2011-10-07 10:45:49.16 Server      * security                       0000000074B40000  0000000074B42FFF  00003000
    2011-10-07 10:45:49.16 Server      * WS2_32                         000007FEFF2C0000  000007FEFF30CFFF  0004d000
    2011-10-07 10:45:49.16 Server      * RsFxFt                         0000000070A40000  0000000070A48FFF  00009000
    2011-10-07 10:45:49.16 Server      * SHELL32                        000007FEFE320000  000007FEFF0A5FFF  00d86000
    2011-10-07 10:45:49.16 Server      * OLEAUT32                       000007FEFF170000  000007FEFF246FFF  000d7000
    2011-10-07 10:45:49.16 Server      * ftimport                       0000000060000000  0000000060024FFF  00025000
    2011-10-07 10:45:49.16 Server      * MSFTE                          000000004DB10000  000000004DEBDFFF  003ae000
    2011-10-07 10:45:49.16 Server      * VERSION                        000007FEFC930000  000007FEFC93BFFF  0000c000
    2011-10-07 10:45:49.16 Server      * dbghelp                        00000000708D0000  0000000070A2DFFF  0015e000
    2011-10-07 10:45:49.16 Server      * WINTRUST                       000007FEFDA70000  000007FEFDAA9FFF  0003a000
    2011-10-07 10:45:49.16 Server      * ncrypt                         000007FEFD220000  000007FEFD26DFFF  0004e000
    2011-10-07 10:45:49.16 Server      * bcrypt                         000007FEFD1F0000  000007FEFD211FFF  00022000
    2011-10-07 10:45:49.16 Server      * mswsock                        000007FEFD000000  000007FEFD053FFF  00054000
    2011-10-07 10:45:49.16 Server      * wship6                         000007FEFD080000  000007FEFD086FFF  00007000
    2011-10-07 10:45:49.16 Server      * wshtcpip                       000007FEFCA00000  000007FEFCA06FFF  00007000
    2011-10-07 10:45:49.16 Server      * ntmarta                        000007FEFC900000  000007FEFC92CFFF  0002d000
    2011-10-07 10:45:49.16 Server      * WLDAP32                        000007FEFDE50000  000007FEFDE9FFFF  00050000
    2011-10-07 10:45:49.16 Server      * ntdsapi                        000007FEFA9E0000  000007FEFAA06FFF  00027000
    2011-10-07 10:45:49.16 Server      * DNSAPI                         000007FEFCE20000  000007FEFCE7AFFF  0005b000
    2011-10-07 10:45:49.16 Server      * rasadhlp                       000007FEFAA20000  000007FEFAA27FFF  00008000
    2011-10-07 10:45:49.16 Server      * fwpuclnt                       000007FEFB860000  000007FEFB8B2FFF  00053000
    2011-10-07 10:45:49.16 Server      * bcryptprimitives               000007FEFCC70000  000007FEFCCBBFFF  0004c000
    2011-10-07 10:45:49.16 Server      * CLBCatQ                        000007FEFF890000  000007FEFF928FFF  00099000
    2011-10-07 10:45:49.16 Server      * sqlncli10                      000000006FB00000  000000006FDB7FFF  002b8000
    2011-10-07 10:45:49.16 Server      * COMCTL32                       000007FEFAE50000  000007FEFAEEFFFF  000a0000
    2011-10-07 10:45:49.16 Server      * COMDLG32                       000007FEFF0B0000  000007FEFF147FFF  00098000
    2011-10-07 10:45:49.16 Server      * SQLNCLIR10                     000000006ECE0000  000000006ED16FFF  00037000
    2011-10-07 10:45:49.16 Server      * netbios                        000007FEF3E70000  000007FEF3E79FFF  0000a000
    2011-10-07 10:45:49.16 Server      * xpsqlbot                       000000006E240000  000000006E247FFF  00008000
    2011-10-07 10:45:49.16 Server      * xpstar                         000000006E1B0000  000000006E237FFF  00088000
    2011-10-07 10:45:49.16 Server      * SQLSCM                         000000006F940000  000000006F94DFFF  0000e000
    2011-10-07 10:45:49.16 Server      * ODBC32                         000007FEF54A0000  000007FEF5550FFF  000b1000
    2011-10-07 10:45:49.16 Server      * ATL80                          0000000074E20000  0000000074E3FFFF  00020000
    2011-10-07 10:45:49.16 Server      * odbcint                        000000006FAA0000  000000006FAD7FFF  00038000
    2011-10-07 10:45:49.16 Server      * clusapi                        000007FEFA8D0000  000007FEFA91FFFF  00050000
    2011-10-07 10:45:49.16 Server      * resutils                       000007FEFA8B0000  000007FEFA8C8FFF  00019000
    2011-10-07 10:45:49.16 Server      * xpstar                         000000006E180000  000000006E1A4FFF  00025000
    2011-10-07 10:45:49.16 Server      * xplog70                        000000006E170000  000000006E17FFFF  00010000
    2011-10-07 10:45:49.16 Server      * xplog70                        000000006E160000  000000006E161FFF  00002000
    2011-10-07 10:45:49.16 Server      * instapi                        0000000019C40000  0000000019C4CFFF  0000d000
    2011-10-07 10:45:49.16 Server      * dsrole                         000007FEFC730000  000007FEFC73BFFF  0000c000
    2011-10-07 10:45:49.16 Server      * apphelp                        000007FEFD6B0000  000007FEFD706FFF  00057000
    2011-10-07 10:45:49.16 Server      * msxmlsql                       000000006EAF0000  000000006EC40FFF  00151000
    2011-10-07 10:45:49.16 Server      * oledb32                        000007FEF3BF0000  000007FEF3CFDFFF  0010e000
    2011-10-07 10:45:49.16 Server      * MSDART                         000007FEF3BC0000  000007FEF3BE7FFF  00028000
    2011-10-07 10:45:49.16 Server      * Comctl32                       000007FEFC170000  000007FEFC363FFF  001f4000
    2011-10-07 10:45:49.16 Server      * OLEDB32R                       000000006E710000  000000006E723FFF  00014000
    2011-10-07 10:45:49.16 Server      * comsvcs                        000007FEF35E0000  000007FEF378FFFF  001b0000
    2011-10-07 10:45:49.16 Server      * db2oledb                       0000000035B80000  0000000035C58FFF  000d9000
    2011-10-07 10:45:49.16 Server      * dbgtrace                       000000006DB50000  000000006DB6EFFF  0001f000
    2011-10-07 10:45:49.16 Server      * SNAREG                         000000006DB40000  000000006DB4CFFF  0000d000
    2011-10-07 10:45:49.16 Server      * RpcRtRemote                    000007FEFD7C0000  000007FEFD7D3FFF  00014000
    2011-10-07 10:45:49.16 Server      * MSDRDA                         000000006D9D0000  000000006DB3EFFF  0016f000
    2011-10-07 10:45:49.16 Server      * MFC80                          000000006D820000  000000006D9BBFFF  0019c000
    2011-10-07 10:45:49.16 Server      * XOLEHLP                        000007FEF38B0000  000007FEF38C3FFF  00014000
    2011-10-07 10:45:49.16 Server      * MFC80ENU                       000000006D810000  000000006D81CFFF  0000d000
    2011-10-07 10:45:49.16 Server      * snanls                         000000006D7D0000  000000006D7F3FFF  00024000
    2011-10-07 10:45:49.16 Server      * TRNSDT                         000000006D7C0000  000000006D7C7FFF  00008000
    2011-10-07 10:45:49.16 Server      * TRNSBIDI                       000000006D7A0000  000000006D7B3FFF  00014000
    2011-10-07 10:45:49.16 Server      * ddmstr                         000000006ED20000  000000006ED2FFFF  00010000
    2011-10-07 10:45:49.16 Server      * dbghelp                        00000000386F0000  000000003884DFFF  0015e000
    2011-10-07 10:45:49.16 Server      *
    2011-10-07 10:45:49.16 Server      *     P1Home: 002E0032002E0031: 
    2011-10-07 10:45:49.16 Server      *     P2Home: 0000000000000030: 
    2011-10-07 10:45:49.16 Server      *     P3Home: 0051005300A400A2: 
    2011-10-07 10:45:49.16 Server      *     P4Home: 0000000060EE3A68:  0050005C003A0043  00720067006F0072  00460020006D0061  00730065006C0069  00630069004D005C  006F0073006F0072 
    2011-10-07 10:45:49.16 Server      *     P5Home: 0000000066CDCD70:  0000000000000000  0000000000000000  0000000000000000  0000000000000000  0000000000000000  0000000000000000 
    2011-10-07 10:45:49.16 Server      *     P6Home: 00000000778D4043:  4100000120C48148  5F5C415D415E415F  9090909090C35B5E  56F3FF9090909090  4156415541544157  00000120EC814857 
    2011-10-07 10:45:49.16 Server      * ContextFlags: 000000000010000F:  0000000000000000  0000000000000000  0000000000000000  0000000000000000  0000000000000000  0000000000000000 
    2011-10-07 10:45:49.16 Server      *      MxCsr: 0000000000001F80: 
    2011-10-07 10:45:49.16 Server      *      SegCs: 0000000000000033: 
    2011-10-07 10:45:49.16 Server      *      SegDs: 000000000000002B: 
    2011-10-07 10:45:49.16 Server      *      SegEs: 000000000000002B: 
    2011-10-07 10:45:49.16 Server      *      SegFs: 0000000000000053: 
    2011-10-07 10:45:49.16 Server      *      SegGs: 000000000000002B: 
    2011-10-07 10:45:49.16 Server      *      SegSs: 000000000000002B: 
    2011-10-07 10:45:49.16 Server      *     EFlags: 0000000000000202: 
    2011-10-07 10:45:49.16 Server      *        Rax: 0000000011541802:  0000000000000000  0000000000000000  0000000000000000  0000000000000000  0000000000000000  0000000000000000 
    2011-10-07 10:45:49.16 Server      *        Rcx: 0000000066CDCB70:  002E0032002E0031  0000000000000030  0051005300A400A2  0000000060EE3A68  0000000066CDCD70  00000000778D4043 
    2011-10-07 10:45:49.16 Server      *        Rdx: 0000000000000000: 
    2011-10-07 10:45:49.16 Server      *        Rbx: 0000000000000000: 
    2011-10-07 10:45:49.16 Server      *        Rsp: 0000000066CDD180:  0000000000000000  0000000000000000  0000000066CDD3B0  000007FFFFE20000  00000000000042AC  0000000000000000 
    2011-10-07 10:45:49.16 Server      *        Rbp: 0000000066CDFE20:  0000000000000000  0000000000000000  0000000000000000  0000000000000000  00000000776F9220  00000000776F9220 
    2011-10-07 10:45:49.17 Server      *        Rsi: 0000000000000000: 
    2011-10-07 10:45:49.17 Server      *        Rdi: 0000000066CDD3B0:  0000000060EE3A60  0000000066CDE5A0  0000000066CE0000  00000000778CF001  000000006DA43D32  0000000066CD0033 
    2011-10-07 10:45:49.17 Server      *         R8: 0000000000000000: 
    2011-10-07 10:45:49.17 Server      *         R9: 0000000000000000: 
    2011-10-07 10:45:49.17 Server      *        R10: 0000000000000000: 
    2011-10-07 10:45:49.17 Server      *        R11: 0000000066CDD990:  002E0032002E0031  0000000000000030  0051005300A400A2  0000000060EE3A68  0000000066CDCD70  00000000778D4043 
    2011-10-07 10:45:49.17 Server      *        R12: 0000000000000001: 
    2011-10-07 10:45:49.17 Server      *        R13: 0000000000000000: 
    2011-10-07 10:45:49.17 Server      *        R14: 0000000066CDE860:  0068005F00780065  006C0064006E0061  00780065005F0065  0074007000650063  0063006E00650020  0074006E0075006F 
    2011-10-07 10:45:49.17 Server      *        R15: 000000000000003F: 
    2011-10-07 10:45:49.17 Server      *        Rip: 000007FEFDA0AA7D:  C3000000C8C48148  9090909090909090  8348EC8348909090  00016A428D0F08F9  74894858245C8948  4D30247C89483824 
    2011-10-07 10:45:49.17 Server      * *******************************************************************************
    2011-10-07 10:45:49.17 Server      * -------------------------------------------------------------------------------
    2011-10-07 10:45:49.17 Server      * Short Stack Dump
    2011-10-07 10:45:49.17 Server      000007FEFDA0AA7D Module(KERNELBASE+000000000000AA7D)
    2011-10-07 10:45:49.17 Server      0000000002BA981D Module(sqlservr+00000000020A981D)
    2011-10-07 10:45:49.17 Server      0000000002BADD3A Module(sqlservr+00000000020ADD3A)
    2011-10-07 10:45:49.17 Server      0000000002BAD89D Module(sqlservr+00000000020AD89D)
    2011-10-07 10:45:49.18 Server      0000000002FED496 Module(sqlservr+00000000024ED496)
    2011-10-07 10:45:49.19 Server      00000000776F9380 Module(kernel32+0000000000099380)
    2011-10-07 10:45:49.19 Server      0000000077916228 Module(ntdll+0000000000096228)
    2011-10-07 10:45:49.19 Server      0000000077894F48 Module(ntdll+0000000000014F48)
    2011-10-07 10:45:49.19 Server      00000000778B4F6D Module(ntdll+0000000000034F6D)
    2011-10-07 10:45:49.19 Server      0000000077895B2C Module(ntdll+0000000000015B2C)
    2011-10-07 10:45:49.19 Server      00000000778960FE Module(ntdll+00000000000160FE)
    2011-10-07 10:45:49.20 Server      000007FEFDA0AA7D Module(KERNELBASE+000000000000AA7D)
    2011-10-07 10:45:49.20 Server      000000007540B164 Module(MSVCR80+000000000003B164)
    2011-10-07 10:45:49.20 Server      000000006DA2C0D0 Module(MSDRDA+000000000005C0D0)
    2011-10-07 10:45:49.20 Server      000000006DA410CB Module(MSDRDA+00000000000710CB)
    2011-10-07 10:45:49.20 Server      000000006DACA853 Module(MSDRDA+00000000000FA853)
    2011-10-07 10:45:49.20 Server      000000006DA43D30 Module(MSDRDA+0000000000073D30)
    2011-10-07 10:45:49.20 Server      000000006DA46EB6 Module(MSDRDA+0000000000076EB6)
    2011-10-07 10:45:49.20 Server      000000006DA47250 Module(MSDRDA+0000000000077250)
    2011-10-07 10:45:49.20 Server      000000006DA3E202 Module(MSDRDA+000000000006E202)
    2011-10-07 10:45:49.20 Server      000000007767F56D Module(kernel32+000000000001F56D)
    2011-10-07 10:45:49.20 Server      00000000778B2CC1 Module(ntdll+0000000000032CC1)
    2011-10-07 10:45:49.20 Server      Stack Signature for the dump is 0x000000011404535D
    2011-10-07 10:45:50.33 Server      External dump process return code 0x20000001.
    External dump process returned no errors.

     

    Thanks in advance

    Regards,
    Mani

     

    Tuesday, October 11, 2011 2:25 PM

Answers

  • No thats a SQL Server 2008 R2 used for building summaries for a dataset that sits on Iseries systems. Yes we have a linked server to DB2 and we are using DB2 OLEDB provider.

    Is that a bad query that ran and caused the Server to Crash ?


    Manishankar Rajagopal | Please mark solved if I've answered your question, vote for it as helpful to help other user's find a solution quicker

    Since the stack is for "msdrda" I would suspect that its due to linked server OleDB driver for DB2. I would sugged to update the drive to latest version and if issue still happens, open support case with Microsoft CSS.
    Balmukund Lakhani | Please mark solved if I've answered your question, vote for it as helpful to help other user's find a solution quicker
    --------------------------------------------------------------------------------
    This posting is provided "AS IS" with no warranties, and confers no rights.
    --------------------------------------------------------------------------------
    My Blog | Team Blog | @Twitter
    • Marked as answer by SP_Mani Tuesday, October 18, 2011 3:52 PM
    Tuesday, October 18, 2011 2:45 AM
    Moderator

All replies

  • As SQL Server was crashed you should be calling customer support and open a case. They should be having right tools to investigate stack dumps.

    Thank you,


    Anup | Forum Support| If you think my suggestion is useful, please rate it as helpful. If it has helped you to resolve the problem, please Mark it as Answer.
    Tuesday, October 11, 2011 5:28 PM
  • This kind of error is almost always a problem with the internal workings of SQL Server.  The first thing you should do it install the latest Service Pack and Cumulative Update to see if the problem is already resolved.

    Please see: http://www.sqlteam.com/article/sql-server-versions

     

    Tuesday, October 11, 2011 9:30 PM
    Moderator
  • Correct,Deploying latest service pack is a good hint to protect your current environment and to do get a RCA for the crash you might have to work with customer support.

     


    Anup | Forum Support| If you think my suggestion is useful, please rate it as helpful. If it has helped you to resolve the problem, please Mark it as Answer.
    Tuesday, October 11, 2011 10:02 PM
  • Could you please share C:\Program Files\Microsoft SQL Server\MSSQL10_50.DWDEV1\MSSQL\LOG\SQLDump0001.mdmp to have a quick look? You can upload the file to any free file hosting site and provide us link in next post.
    Balmukund Lakhani | Please mark solved if I've answered your question, vote for it as helpful to help other user's find a solution quicker
    --------------------------------------------------------------------------------
    This posting is provided "AS IS" with no warranties, and confers no rights.
    --------------------------------------------------------------------------------
    My Blog | Team Blog | @Twitter
    Wednesday, October 12, 2011 2:38 AM
    Moderator
  • I have uploaded the file to the below share location

    https://lhe.sharefile.com/?cmd=d&id=f11990d415594fb2

    Let me know if it is accessible.

    Thanks again.

    Regards,

    Mani

    Wednesday, October 12, 2011 7:03 PM
  •  HI Balmukund

    Can I ask a favor if you can take a look at my sql dump file from the link below and see if you can tell what might have caused the crash.

    WE have sql 2008 r2 sp1 and windows 2008 r2 sp1

     

    thanks

    Conan

     

    https://docs.google.com/viewer?a=v&pid=explorer&chrome=true&srcid=0B08LkCj8yW_IYmY1YmU0YTMtOTFhZS00MDcxLWI1MzEtNDQ0MDE5MDdjZDM0&hl=en_US 

    Saturday, October 15, 2011 4:06 PM
  •  HI Balmukund

    Can I ask a favor if you can take a look at my sql dump file from the link below and see if you can tell what might have caused the crash.

    WE have sql 2008 r2 sp1 and windows 2008 r2 sp1

     

    thanks

    Conan

     

    https://docs.google.com/viewer?a=v&pid=explorer&chrome=true&srcid=0B08LkCj8yW_IYmY1YmU0YTMtOTFhZS00MDcxLWI1MzEtNDQ0MDE5MDdjZDM0&hl=en_US 


     

    0:104> lmv msqlservr

        Loaded symbol image file: sqlservr.exe
        Image name: sqlservr.exe
        ProductVersion:   10.50.2500.0
        ProductName:      Microsoft SQL Server

     

    0:104> kL 100
     ntdll!ZwWriteFileGather+0xa
     KERNELBASE!WriteFileGather+0x8d
     kernel32!WriteFileGatherStub+0x13                                             << Disk Operation started here in Kernel Mode
     sqlservr!DiskGatherWriteAsync+0xee
     sqlservr!FCB::GatherWrite+0x1db
     sqlservr!Bob::IssueWrite+0x237
     sqlservr!BobMgr::GetBuf+0x85
     sqlservr!get_bob_buf+0x21

     sqlservr!addrun+0x187

     sqlservr!sort_addpage+0x120

     sqlservr!soMinimizeMemory+0x34

     sqlservr!sort_endinput+0x86

     sqlservr!RowsetSorted::EndInputPhase+0x63

     sqlservr!CQScanIndexSortNew::UlEndSortInput+0x73

     sqlservr!CQScanSortNew::TransitionToOutputPhase+0x2a

     sqlservr!CQScanSortNew::PushClose+0x36

     sqlservr!CQScanIndexSortNew::BuildSortTable+0x209

     sqlservr!CQScanIndexSortNew::OpenHelper+0x7a

     sqlservr!CQScanNew::OpenHelper+0x34

     sqlservr!CQScanIndexNew::Open+0x36

     sqlservr!CQScanNew::OpenHelper+0x34

     sqlservr!CQScanXProducerNew::Open+0x35

     sqlservr!FnProducerOpen+0x45

     sqlservr!FnProducerThread+0x4df

     sqlservr!SubprocEntrypoint+0x794                                                                <<< This is parallel thread

     sqlservr!SOS_Task::Param::Execute+0x12a

     sqlservr!SOS_Scheduler::RunTask+0x96

     sqlservr!SOS_Scheduler::ProcessTasks+0x128

     sqlservr!SchedulerManager::WorkerEntryPoint+0x2d2

     sqlservr!SystemThread::RunWorker+0xcc

     sqlservr!SystemThreadDispatcher::ProcessWorker+0x2db

     sqlservr!SchedulerManager::ThreadEntryPoint+0x173

     msvcr80!_callthreadstartex+0x17

     msvcr80!_threadstartex+0x84

     kernel32!BaseThreadInitThunk+0xd

     ntdll!RtlUserThreadStart+0x1d

    from the above I would suspect that there are some issues with the underlying hardware subsystem. Can you check ERRORLOG if you see any messages like below

    SQL Server has encountered xxx occurrence(s) of I/O requests taking longer than 15 seconds to complete on file ....


    Balmukund Lakhani | Please mark solved if I've answered your question, vote for it as helpful to help other user's find a solution quicker
    --------------------------------------------------------------------------------
    This posting is provided "AS IS" with no warranties, and confers no rights.
    --------------------------------------------------------------------------------
    My Blog | Team Blog | @Twitter
    Monday, October 17, 2011 9:02 AM
    Moderator
  • I have uploaded the file to the below share location

    https://lhe.sharefile.com/?cmd=d&id=f11990d415594fb2

    Let me know if it is accessible.

    Thanks again.

    Regards,

    Mani


    Hi Mani
    below is the stack which I see in the dump.

     

     ntdll!ZwWaitForSingleObject+0xa

     KERNELBASE!WaitForSingleObjectEx+0x79

     sqlservr!CDmpDump::DumpInternal+0x4d9

     sqlservr!CDmpDump::DumpFilter+0xbc

     sqlservr!`CDmpDump::Dump'::`1'::filt$0+0x21

     msvcr80!__C_specific_handler+0x96

     ntdll!RtlpExecuteHandlerForException+0xd

     ntdll!RtlDispatchException+0x3cb

     ntdll!RtlRaiseException+0x221

     KERNELBASE!RaiseException+0x39

     sqlservr!CDmpDump::Dump+0x7c

     sqlservr!SQLDumperLibraryInvoke+0x1a0

     sqlservr!CImageHelper::DoMiniDump+0x3d4

     sqlservr!stackTrace+0x82b

     sqlservr!stackTraceCallBack+0x49

     sqlservr!ex_handle_except+0x106

     kernel32!UnhandledExceptionFilter+0x160

     ntdll!RtlUserThreadStart$filt$0+0x45

     ntdll!__C_specific_handler+0x8c

     ntdll!RtlpExecuteHandlerForException+0xd

     ntdll!RtlDispatchException+0x3cb

     ntdll!RtlRaiseException+0x221

     KERNELBASE!RaiseException+0x39

     msvcr80!_CxxThrowException+0xc4

     msdrda!ddm::FetchCallback::Resize+0x50

     msdrda!ddm::IChunkedBuffer::Resize+0x2b

     msdrda!ddm::ChunkedSwapBuffer::ReadByte+0x103

     msdrda!ddm::AdjustableBuffer::ReadByte+0x20

     msdrda!ddm::RowReader::ProcessRow+0x1a6

     msdrda!ddm::RowReader::GetRows+0x260

     msdrda!ddm::PrefetchReceiveThread+0x222

     kernel32!BaseThreadInitThunk+0xd

     ntdll!RtlUserThreadStart+0x1d

    Do you have any linked server to DB2?
    Whats the Provider?
    Is this a BizTalk Server?



    Balmukund Lakhani | Please mark solved if I've answered your question, vote for it as helpful to help other user's find a solution quicker
    --------------------------------------------------------------------------------
    This posting is provided "AS IS" with no warranties, and confers no rights.
    --------------------------------------------------------------------------------
    My Blog | Team Blog | @Twitter
    Monday, October 17, 2011 9:13 AM
    Moderator
  • No thats a SQL Server 2008 R2 used for building summaries for a dataset that sits on Iseries systems. Yes we have a linked server to DB2 and we are using DB2 OLEDB provider.

    Is that a bad query that ran and caused the Server to Crash ?


    Manishankar Rajagopal | Please mark solved if I've answered your question, vote for it as helpful to help other user's find a solution quicker
    Monday, October 17, 2011 3:22 PM
  • HI Balmukund

    Thank you for your very quick response.

    In answer to your question  yes I do see the "SQL Server has encountered xxx occurrence(s) of I/O requests taking longer than 15 seconds to complete " message in the error log?

    Any idea what you think this all means? Did the disk  cause the sql  to crash ?

     

    thanks

    Conan



    • Edited by Conan11 Tuesday, October 18, 2011 12:00 AM
    Monday, October 17, 2011 11:27 PM
  • HI Balmukund

    Thank you for your very quick response.

    In answer to your question  yes I do see the "SQL Server has encountered xxx occurrence(s) of I/O requests taking longer than 15 seconds to complete " message in the error log?

    Any idea what you think this all means? Did the disk  cause the sql  to crash ?

    thanks

    Conan

    Refer http://blogs.msdn.com/b/sqlserverstorageengine/archive/2006/06/21/642314.aspx for explaination about warning message.
    You are correct that "slow" disk caused "Deadlock Schedulers" and no work was moving forward in SQL and it appeared to be hung. Without ErrorLog I would not be able to say it was "crashed"


    Balmukund Lakhani | Please mark solved if I've answered your question, vote for it as helpful to help other user's find a solution quicker
    --------------------------------------------------------------------------------
    This posting is provided "AS IS" with no warranties, and confers no rights.
    --------------------------------------------------------------------------------
    My Blog | Team Blog | @Twitter
    Tuesday, October 18, 2011 2:42 AM
    Moderator
  • No thats a SQL Server 2008 R2 used for building summaries for a dataset that sits on Iseries systems. Yes we have a linked server to DB2 and we are using DB2 OLEDB provider.

    Is that a bad query that ran and caused the Server to Crash ?


    Manishankar Rajagopal | Please mark solved if I've answered your question, vote for it as helpful to help other user's find a solution quicker

    Since the stack is for "msdrda" I would suspect that its due to linked server OleDB driver for DB2. I would sugged to update the drive to latest version and if issue still happens, open support case with Microsoft CSS.
    Balmukund Lakhani | Please mark solved if I've answered your question, vote for it as helpful to help other user's find a solution quicker
    --------------------------------------------------------------------------------
    This posting is provided "AS IS" with no warranties, and confers no rights.
    --------------------------------------------------------------------------------
    My Blog | Team Blog | @Twitter
    • Marked as answer by SP_Mani Tuesday, October 18, 2011 3:52 PM
    Tuesday, October 18, 2011 2:45 AM
    Moderator
  • One last question , in the article it says " have a look at the physical disk counters in sysmon to ensure that the disks are servicing IOs in a reasonable period of time. "

     Which disk counters do you recommend I look at ? some of our disk are raid 5 and some raid 10.

     

    Thanks again,

    Conan

     

    UPDATE... I just noticed if I turn of  parallelism those errors go away..Please advise..

     

     

     

     

     

    • Edited by Conan11 Tuesday, October 18, 2011 5:32 PM
    Tuesday, October 18, 2011 1:16 PM
  • One last question , in the article it says " have a look at the physical disk counters in sysmon to ensure that the disks are servicing IOs in a reasonable period of time. "

     Which disk counters do you recommend I look at ? some of our disk are raid 5 and some raid 10.

     

    Thanks again,

    Conan

     

    UPDATE... I just noticed if I turn of  parallelism those errors go away..Please advise..


    What was the maxdop setting and how many processors you have?
    Balmukund Lakhani | Please mark solved if I've answered your question, vote for it as helpful to help other user's find a solution quicker
    --------------------------------------------------------------------------------
    This posting is provided "AS IS" with no warranties, and confers no rights.
    --------------------------------------------------------------------------------
    My Blog | Team Blog | @Twitter
    Tuesday, October 18, 2011 10:08 PM
    Moderator
  • HI Balmukund

    the setting was the default of  0. Now I have it to 1.

    The server has a dual quad processor , In task manager I see  16 processors

     

    thanks

    Conan



    • Edited by Conan11 Wednesday, October 19, 2011 2:29 AM
    Tuesday, October 18, 2011 11:32 PM
  • HI Balmukund

    the setting was the default of  0. Now I have it to 1.

    The server has a dual quad processor , In task manager I see  16 processors

     thanks

    Conan




    quite possible that there might be a sub-optimal plan generated with parallelism. Refer below.

    http://support.microsoft.com/kb/329204
    General guidelines to use to configure the MAXDOP option

    Balmukund Lakhani | Please mark solved if I've answered your question, vote for it as helpful to help other user's find a solution quicker
    --------------------------------------------------------------------------------
    This posting is provided "AS IS" with no warranties, and confers no rights.
    --------------------------------------------------------------------------------
    My Blog | Team Blog | @Twitter
    Wednesday, October 19, 2011 6:04 AM
    Moderator
  • One last question , in the article it says " have a look at the physical disk counters in sysmon to ensure that the disks are servicing IOs in a reasonable period of time. "

     Which disk counters do you recommend I look at ? some of our disk are raid 5 and some raid 10.

     

    Thanks again,
    Conan

    UPDATE... I just noticed if I turn of  parallelism those errors go away..Please advise.. 


    For disk counters, refer http://msdn.microsoft.com/en-us/library/dd672789(v=SQL.100).aspx "I/O Bottlenecks" section.

    Balmukund Lakhani | Please mark solved if I've answered your question, vote for it as helpful to help other user's find a solution quicker
    --------------------------------------------------------------------------------
    This posting is provided "AS IS" with no warranties, and confers no rights.
    --------------------------------------------------------------------------------
    My Blog | Team Blog | @Twitter
    Wednesday, October 19, 2011 6:08 AM
    Moderator
  • Balmukund Thanks a lot for your help!!!!
    Thursday, October 20, 2011 1:53 AM
  • Balamukund,

    Thanks for your help, if this becomes a perpetual problem I am going to call up PSS

     


    Manishankar Rajagopal | Please mark solved if I've answered your question, vote for it as helpful to help other user's find a solution quicker
    Thursday, October 20, 2011 4:47 PM
  • Balamukund,

    Thanks for your help, if this becomes a perpetual problem I am going to call up PSS

     


    Manishankar Rajagopal | Please mark solved if I've answered your question, vote for it as helpful to help other user's find a solution quicker

    Sure! Good choice!
    Balmukund Lakhani | Please mark solved if I've answered your question, vote for it as helpful to help other user's find a solution quicker
    --------------------------------------------------------------------------------
    This posting is provided "AS IS" with no warranties, and confers no rights.
    --------------------------------------------------------------------------------
    My Blog | Team Blog | @Twitter
    Thursday, October 20, 2011 4:49 PM
    Moderator
  • Sure! Good choice!
    Balmukund Lakhani | Please mark solved if I've answered your question, vote for it as helpful to help other user's find a solution quicker
    --------------------------------------------------------------------------------
    This posting is provided "AS IS" with no warranties, and confers no rights.
    --------------------------------------------------------------------------------
    My Blog | Team Blog | @Twitter

    Please let me know the SR number via email to my email id blakhani (at) koolmail (dot) in
    Balmukund Lakhani | Please mark solved if I've answered your question, vote for it as helpful to help other user's find a solution quicker
    --------------------------------------------------------------------------------
    This posting is provided "AS IS" with no warranties, and confers no rights.
    --------------------------------------------------------------------------------
    My Blog | Team Blog | @Twitter
    Thursday, October 20, 2011 4:50 PM
    Moderator
  • action plan from my side would be that you need to enable trace flag 2551 and wait for next occurence.


    Balmukund Lakhani | Please mark solved if I've answered your question, vote for it as helpful to help other user's find a solution quicker
    --------------------------------------------------------------------------------
    This posting is provided "AS IS" with no warranties, and confers no rights.
    --------------------------------------------------------------------------------
    My Blog | Team Blog | @Twitter
    Thursday, October 20, 2011 4:51 PM
    Moderator
  • Hello My SQL server just crashed this afternoon can you take a look of my mdmp file and tell me what would happened 

    this is the address where I Upload the file

    http://www.filefactory.com/file/1oc85w4e0z5l/n/SQLDump0001.mdmp

    thanks

    Tuesday, May 08, 2012 10:27 PM