none
getting exception Insufficient memory RRS feed

  • Question


  • In production, we are running multiple wcf services with service bus concept. and now in production environment process is crashing because it is receiving 130 mb message and at that time only 120 mb memory is available. so due to this it is throwing Insufficient memory and process is crashing. 

    Below is windbg stack trace which i am getting :

    0:052> !pe

    Exception object: 0152b790
    Exception type:   System.InsufficientMemoryException
    Message:          Failed to allocate a managed memory buffer of 134217728 bytes. The amount of available memory may be low.
    InnerException:   System.OutOfMemoryException, Use !PrintException 95f344cb to see more.
    StackTrace (generated):
        SP       IP       Function
        14A8E660 0B35BD67 UNKNOWN!System.Runtime.Fx.AllocateByteArray(Int32)+0x87
        14A8E73C 0B35BC56 UNKNOWN!System.Runtime.InternalBufferManager+PooledBufferManager.TakeBuffer(Int32)+0x66
        14A8E750 0B35BB52 UNKNOWN!System.ServiceModel.Channels.BufferManagerOutputStream.Init(Int32, Int32, Int32, System.ServiceModel.Channels.BufferManager)+0x32
        14A8E76C 0B35B98E UNKNOWN!System.ServiceModel.Channels.BufferedMessageWriter.WriteMessage(System.ServiceModel.Channels.Message, System.ServiceModel.Channels.BufferManager, Int32, Int32)+0x6e
        14A8E7CC 0B35B031 UNKNOWN!System.ServiceModel.Channels.TextMessageEncoderFactory+TextMessageEncoder.WriteMessage(System.ServiceModel.Channels.Message, Int32, System.ServiceModel.Channels.BufferManager, Int32)+0x91
        14A8E80C 0B35AF37 UNKNOWN!System.ServiceModel.Channels.HttpOutput.SerializeBufferedMessage(System.ServiceModel.Channels.Message)+0x47
        14A8E834 0B357707 UNKNOWN!System.ServiceModel.Channels.HttpOutput.Send(System.TimeSpan)+0xf7
        14A8E864 0E7091CA UNKNOWN!System.ServiceModel.Channels.HttpRequestContext.OnReply(System.ServiceModel.Channels.Message, System.TimeSpan)+0x7a
        14A8E8BC 0B8EFFC2 UNKNOWN!System.ServiceModel.Channels.RequestContextBase.Reply(System.ServiceModel.Channels.Message, System.TimeSpan)+0x92
        14A8E8FC 0F87D5BF UNKNOWN!System.ServiceModel.Security.SecuritySessionServerSettings+ServerSecuritySessionChannel.SendMessage(System.ServiceModel.Channels.RequestContext, System.ServiceModel.Channels.Message, System.TimeSpan)+0x8f
        14A8E93C 0F87D468 UNKNOWN!System.ServiceModel.Security.SecuritySessionServerSettings+ServerSecuritySessionChannel.SendCloseResponse(System.ServiceModel.Channels.RequestContext, System.ServiceModel.Channels.Message, System.TimeSpan)+0x68
        14A8E994 0F87D1CA UNKNOWN!System.ServiceModel.Security.SecuritySessionServerSettings+ServerSecuritySimplexSessionChannel.SendCloseResponseOnCloseIfRequired(System.TimeSpan)+0x8a
        14A8E9F4 0F87CFF3 UNKNOWN!System.ServiceModel.Security.SecuritySessionServerSettings+ServerSecuritySimplexSessionChannel.OnClose(System.TimeSpan)+0x53
        14A8EA44 0B305EDD UNKNOWN!System.ServiceModel.Channels.CommunicationObject.Close(System.TimeSpan)+0x1ed
        14A8EAB8 0B30635B UNKNOWN!System.ServiceModel.Channels.ServiceChannel.OnClose(System.TimeSpan)+0xab
        14A8EAF4 0B305EDD UNKNOWN!System.ServiceModel.Channels.CommunicationObject.Close(System.TimeSpan)+0x1ed
        14A8EB68 0B8FD788 UNKNOWN!System.ServiceModel.Channels.ServiceChannel.DecrementActivity()+0xc8
        14A8EBA0 0B8FD697 UNKNOWN!System.ServiceModel.Channels.ServiceChannel.HandleReceiveComplete(System.ServiceModel.Channels.RequestContext)+0xd7
        14A8EBD8 0B94505C UNKNOWN!System.ServiceModel.Dispatcher.ChannelHandler.HandleReceiveComplete(System.ServiceModel.Channels.RequestContext)+0x5c
        14A8EC10 0B944CF1 UNKNOWN!System.ServiceModel.Dispatcher.ChannelHandler.AsyncMessagePump(System.IAsyncResult)+0x51
        14A8EC24 0E6E2B44 UNKNOWN!System.ServiceModel.Dispatcher.ChannelHandler.OnContinueAsyncReceive(System.Object)+0x44
        14A8EC34 0B10B9A9 UNKNOWN!System.Runtime.IOThreadScheduler+ScheduledOverlapped.IOCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)+0x79
        14A8EC74 0B10B8BA UNKNOWN!System.Runtime.Fx+IOCompletionThunk.UnhandledExceptionFrame(UInt32, UInt32, System.Threading.NativeOverlapped*)+0x4a
        14A8ECA4 72665C64 mscorlib_ni!System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)+0x74

    StackTraceString: <none>
    HResult: 8013153d
    There are nested exceptions on this thread. Run with -nested for details

    0:052> !mk -a -p
    Thread 52:
            SP       IP
    00:U 14a8ea4c 76d1c41f KERNELBASE!RaiseException+0x58
    01:U 14a8eaa4 73342b54 clr!RaiseTheExceptionInternalOnly+0x276
    02:U 14a8eb34 73346057 clr!IL_Rethrow+0xe8
    03:M 14a8ebd4 0b10b8ba System.Runtime.Fx+IOCompletionThunk.UnhandledExceptionFrame(UInt32, UInt32, System.Threading.NativeOverlapped*)(+0x20 IL,+0x4a Native)
         [A0]:this: <UNAVAILABLE> (IOCompletionThunk)
         [A1]:error: <UNAVAILABLE> (System.UInt32)
         [A2]:bytesRead: <UNAVAILABLE> (System.UInt32)
         [A3]:nativeOverlapped: <UNAVAILABLE> (System.Threading.NativeOverlapped*)
         [L0]: <UNAVAILABLE> (System.Exception)
    04:M 14a8ec9c 72665c64 System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)(+0x78 IL,+0x74 Native)
         [A0]:errorCode: <UNAVAILABLE> (System.UInt32)
         [A1]:numBytes: <UNAVAILABLE> (System.UInt32)
         [A2]:pOVERLAP: <UNAVAILABLE> (System.Threading.NativeOverlapped*)
         [L0]: <UNAVAILABLE> (System.Threading.Overlapped)
         [L1]: <UNAVAILABLE> (System.Threading._IOCompletionCallback)
         [L2]: <UNAVAILABLE> (System.Threading.IOCompletionCallback)
         [L3]: <UNAVAILABLE> (System.Threading.ExecutionContext)
    05:U 14a8ecdc 732621bb clr!CallDescrWorker+0x33
    06:U 14a8ecf0 7328a553 clr!CallDescrWorkerWithHandler+0x8e
    07:U 14a8ed6c 7328cb1b clr!DispatchCallBody+0x20
    08:U 14a8ed8c 7328cb94 clr!DispatchCallDebuggerWrapper+0x75
    09:U 14a8ee20 7328cc03 clr!DispatchCallNoEH+0x53
    0a:U 14a8ee54 73500292 clr!BindIoCompletionCallBack_Worker+0xc1
    0b:U 14a8eec0 73415702 clr!Thread::DoExtraWorkForFinalizer+0x114
    0c:U 14a8eed4 73415784 clr!Thread::ShouldChangeAbortToUnload+0x101
    0d:U 14a8ef84 7341583f clr!Thread::ShouldChangeAbortToUnload+0x399
    0e:U 14a8efe4 73415db5 clr!Thread::RaiseCrossContextException+0x3f8
    0f:U 14a8eff0 73568174 clr!Thread::DoADCallBack+0x358
    10:U 14a8f0d0 73415de2 clr!Thread::DoExtraWorkForFinalizer+0xfa
    11:U 14a8f0f0 73415784 clr!Thread::ShouldChangeAbortToUnload+0x101
    12:U 14a8f1a0 7341583f clr!Thread::ShouldChangeAbortToUnload+0x399
    13:U 14a8f200 734158d9 clr!Thread::ShouldChangeAbortToUnload+0x43a
    14:U 14a8f224 733270d5 clr!ManagedThreadBase::ThreadPool+0x15
    15:U 14a8f23c 73500d08 clr!BindIoCompletionCallbackStubEx+0xa3
    16:U 14a8f298 73500d42 clr!BindIoCompletionCallbackStub+0x15
    17:U 14a8f2b0 73411ac0 clr!ThreadpoolMgr::CompletionPortThreadStart+0x4e3
    18:U 14a8f30c 734176e0 clr!Thread::intermediateThreadProc+0x4b
    19:U 14a8fca8 76e233aa kernel32!BaseThreadInitThunk+0xe
    1a:U 14a8fcb4 77459ef2 ntdll!__RtlUserThreadStart+0x70
    1b:U 14a8fcf4 77459ec5 ntdll!_RtlUserThreadStart+0x1b

    0:052> !muf 0b10b8ba
    IOCompletionThunk.UnhandledExceptionFrame(uint, uint, System.Threading.NativeOverlapped*): void
    Source information not available.
        loc 0:System.Exception

            0b10b870 55              push    ebp
            0b10b871 8bec            mov     ebp,esp
            0b10b873 57              push    edi
            0b10b874 56              push    esi
            0b10b875 53              push    ebx
            0b10b876 83ec14          sub     esp,14h
            0b10b879 33c0            xor     eax,eax
            0b10b87b 8945e0          mov     dword ptr [ebp-20h],eax
            0b10b87e 8945e4          mov     dword ptr [ebp-1Ch],eax
            0b10b881 8945e8          mov     dword ptr [ebp-18h],eax
            0b10b884 8945ec          mov     dword ptr [ebp-14h],eax
            0b10b887 33c0            xor     eax,eax
            0b10b889 8945e8          mov     dword ptr [ebp-18h],eax
            0b10b88c 8bf1            mov     esi,ecx
            0b10b88e 8bfa            mov     edi,edx
        IL_0000: call System.Runtime.CompilerServices.RuntimeHelpers::PrepareConstrainedRegions
            0b10b890 e885171868      call    clr!LeaveRuntimeStackProbeOnly (7328d01a)
    .try {
        IL_0005: ldarg.0  (error)
        IL_0006: ldfld IOCompletionThunk::callback
        IL_000b: ldarg.1  (bytesRead)
        IL_000c: ldarg.2  (nativeOverlapped)
        IL_000d: ldarg.3 
        IL_000e: callvirt System.Threading.IOCompletionCallback::Invoke
        IL_0013: leave.s IL_0022
            0b10b895 8b4e04          mov     ecx,dword ptr [esi+4]
            0b10b898 ff750c          push    dword ptr [ebp+0Ch]
            0b10b89b ff7508          push    dword ptr [ebp+8]
            0b10b89e 8bd7            mov     edx,edi
            0b10b8a0 8b410c          mov     eax,dword ptr [ecx+0Ch]
            0b10b8a3 8b4904          mov     ecx,dword ptr [ecx+4]
            0b10b8a6 ffd0            call    eax
            0b10b8a8 eb15            jmp     0b10b8bf
    } // END TRY (IL_0005)
    .catch {
        IL_0015: stloc.0 
            0b10b8aa 8bc8            mov     ecx,eax
        IL_0016: ldloc.0 
        IL_0017: call System.Runtime.Fx::HandleAtThreadBase(System.Exception)
        IL_001c: brtrue.s IL_0020
            0b10b8ac e89ffeffff      call    0b10b750  [System.Runtime.Fx.HandleAtThreadBase(System.Exception), MD=09393234]
            0b10b8b1 85c0            test    eax,eax
            0b10b8b3 7505            jne     0b10b8ba
        IL_001e: rethrow 
            0b10b8b5 e8eca62368      call    clr!IL_Rethrow (73345fa6)
        IL_0020: leave.s IL_0022
            0b10b8ba e8f9691568      call    clr!JIT_EndCatch (732622b8)
    } // END CATCH (IL_0015)
        IL_0022: ret 
            0b10b8bf 8d65f4          lea     esp,[ebp-0Ch]
            0b10b8c2 5b              pop     ebx
            0b10b8c3 5e              pop     esi
            0b10b8c4 5f              pop     edi
            0b10b8c5 5d              pop     ebp
            0b10b8c6 c20800          ret     8
    Invalid offset (704e033a) passed to ILDecoder.
    0:052> !muf 72665c64
    System.Threading._IOCompletionCallback.PerformIOCompletionCallback(uint, uint, System.Threading.NativeOverlapped*): void
    Source information not available.
        loc 0:System.Threading.Overlapped
        loc 1:System.Threading._IOCompletionCallback
        loc 2:System.Threading.IOCompletionCallback
        loc 3:System.Threading.ExecutionContext

            72665bf0 55              push    ebp
            72665bf1 8bec            mov     ebp,esp
            72665bf3 57              push    edi
            72665bf4 56              push    esi
            72665bf5 53              push    ebx
            72665bf6 83ec24          sub     esp,24h
            72665bf9 8bf1            mov     esi,ecx
            72665bfb 8d7dd0          lea     edi,[ebp-30h]
            72665bfe b909000000      mov     ecx,9
            72665c03 33c0            xor     eax,eax
            72665c05 f3ab            rep stos dword ptr es:[edi]
            72665c07 8bce            mov     ecx,esi
            72665c09 33c0            xor     eax,eax
            72665c0b 8945e4          mov     dword ptr [ebp-1Ch],eax
            72665c0e 894dd8          mov     dword ptr [ebp-28h],ecx
            72665c11 8955d4          mov     dword ptr [ebp-2Ch],edx
            72665c14 bab3010000      mov     edx,1B3h
            72665c19 b901000000      mov     ecx,1
            72665c1e e83d04f4ff      call    mscorlib_ni+0x1d6060 (725a6060)
            72665c23 8d55f0          lea     edx,[ebp-10h]
            72665c26 b964994372      mov     ecx,offset mscorlib_ni+0x69964 (72439964)
            72665c2b e87804f4ff      call    mscorlib_ni+0x1d60a8 (725a60a8)
        IL_0000: ldarg.2  (pOVERLAP)
        IL_0001: call System.Threading.OverlappedData::GetOverlappedFromNative(System.Threading.NativeOverlapped*)
        IL_0006: ldfld System.Threading.OverlappedData::m_overlapped
        IL_000b: stloc.0 
        IL_000c: ldloc.0 
        IL_000d: callvirt System.Threading.Overlapped::get_iocbHelper()
        IL_0012: stloc.1 
            72665c30 8b4d08          mov     ecx,dword ptr [ebp+8]
            72665c33 e85014f4ff      call    mscorlib_ni+0x1d7088 (725a7088)
            72665c38 8b7010          mov     esi,dword ptr [eax+10h]
            72665c3b 8bce            mov     ecx,esi
            72665c3d 3909            cmp     dword ptr [ecx],ecx
            72665c3f e86c000000      call    mscorlib_ni+0x295cb0 (72665cb0)  [System.Threading.Overlapped.get_iocbHelper(), MD=724f1ca4]
            72665c44 8bf8            mov     edi,eax
        IL_0013: ldloc.1 
        IL_0014: brfalse.s IL_002c
        IL_0016: ldloc.1 
        IL_0017: ldfld System.Threading._IOCompletionCallback::_executionContext
        IL_001c: brfalse.s IL_002c
        IL_001e: ldloc.1 
        IL_001f: ldfld System.Threading._IOCompletionCallback::_executionContext
        IL_0024: ldc.i4.1 
        IL_0025: callvirt System.Threading.ExecutionContext::IsDefaultFTContext(bool)
        IL_002a: brfalse.s IL_003e
        IL_002c: ldloc.0 
        IL_002d: callvirt System.Threading.Overlapped::get_UserCallback()
        IL_0032: stloc.2 
            72665c46 85ff            test    edi,edi
            72665c48 7539            jne     mscorlib_ni+0x295c83 (72665c83)
            72665c4a 8bce            mov     ecx,esi
            72665c4c e87f000000      call    mscorlib_ni+0x295cd0 (72665cd0)
        IL_0033: ldloc.2 
        IL_0034: ldarg.0  (errorCode)
        IL_0035: ldarg.1  (numBytes)
        IL_0036: ldarg.2  (pOVERLAP)
        IL_0037: callvirt System.Threading.IOCompletionCallback::Invoke(uint, uint, System.Threading.NativeOverlapped*)
        IL_003c: br.s IL_0078
        IL_003e: ldloc.1 
        IL_003f: ldarg.0  (errorCode)
        IL_0040: stfld System.Threading._IOCompletionCallback::_errorCode
        IL_0045: ldloc.1 
        IL_0046: ldarg.1  (numBytes)
        IL_0047: stfld System.Threading._IOCompletionCallback::_numBytes
        IL_004c: ldloc.1 
        IL_004d: ldarg.2  (pOVERLAP)
        IL_004e: stfld System.Threading._IOCompletionCallback::_pOVERLAP
        IL_0053: ldloc.1 
        IL_0054: ldfld System.Threading._IOCompletionCallback::_executionContext
        IL_0059: callvirt System.Threading.ExecutionContext::CreateCopy()
        IL_005e: stloc.3 
    .try {
        IL_005f: ldloc.3 
        IL_0060: ldsfld System.Threading._IOCompletionCallback::_ccb
        IL_0065: ldloc.1 
        IL_0066: ldc.i4.1 
        IL_0067: call System.Threading.ExecutionContext::Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, object, bool)
        IL_006c: leave.s IL_0078
    } // END TRY (IL_005f)
    .finally {
        IL_006e: ldloc.3 
        IL_006f: brfalse.s IL_0077
        IL_0071: ldloc.3 
        IL_0072: callvirt System.IDisposable::Dispose()
        IL_0077: endfinally 
            72665c51 ff75d4          push    dword ptr [ebp-2Ch]
            72665c54 ff7508          push    dword ptr [ebp+8]
            72665c57 8b55d8          mov     edx,dword ptr [ebp-28h]
            72665c5a 8bc8            mov     ecx,eax
            72665c5c 8b410c          mov     eax,dword ptr [ecx+0Ch]
            72665c5f 8b4904          mov     ecx,dword ptr [ecx+4]
            72665c62 ffd0            call    eax
    } // END FINALLY (IL_006e)
        IL_0078: ldarga.s 2 (pOVERLAP)
        IL_007a: ldarga.s 0 (errorCode)
        IL_007c: ldarga.s 1 (numBytes)
        IL_007e: call System.Threading.OverlappedData::CheckVMForIOPacket(System.Threading.NativeOverlapped*&, uint&, uint&)
            72665c64 8d45d4          lea     eax,[ebp-2Ch]
            72665c67 50              push    eax
            72665c68 8d4d08          lea     ecx,[ebp+8]
            72665c6b 8d55d8          lea     edx,[ebp-28h]
            72665c6e e82514f4ff      call    mscorlib_ni+0x1d7098 (725a7098)
        IL_0083: ldarg.2  (pOVERLAP)
        IL_0084: ldc.i4.0 
        IL_0085: conv.u 
        IL_0086: bne.un IL_0000
            72665c73 837d0800        cmp     dword ptr [ebp+8],0
            72665c77 75b7            jne     mscorlib_ni+0x295c30 (72665c30)
        IL_008b: ret 
            72665c79 8d65f4          lea     esp,[ebp-0Ch]
            72665c7c 5b              pop     ebx
            72665c7d 5e              pop     esi
            72665c7e 5f              pop     edi
            72665c7f 5d              pop     ebp
            72665c80 c20400          ret     4
        IL_0016: ldloc.1 
        IL_0017: ldfld System.Threading._IOCompletionCallback::_executionContext
        IL_001c: brfalse.s IL_002c
        IL_001e: ldloc.1 
        IL_001f: ldfld System.Threading._IOCompletionCallback::_executionContext
        IL_0024: ldc.i4.1 
        IL_0025: callvirt System.Threading.ExecutionContext::IsDefaultFTContext(bool)
        IL_002a: brfalse.s IL_003e
        IL_002c: ldloc.0 
        IL_002d: callvirt System.Threading.Overlapped::get_UserCallback()
        IL_0032: stloc.2 
        IL_0033: ldloc.2 
        IL_0034: ldarg.0  (errorCode)
        IL_0035: ldarg.1  (numBytes)
        IL_0036: ldarg.2  (pOVERLAP)
        IL_0037: callvirt System.Threading.IOCompletionCallback::Invoke(uint, uint, System.Threading.NativeOverlapped*)
        IL_003c: br.s IL_0078
        IL_003e: ldloc.1 
        IL_003f: ldarg.0  (errorCode)
        IL_0040: stfld System.Threading._IOCompletionCallback::_errorCode
        IL_0045: ldloc.1 
        IL_0046: ldarg.1  (numBytes)
        IL_0047: stfld System.Threading._IOCompletionCallback::_numBytes
        IL_004c: ldloc.1 
        IL_004d: ldarg.2  (pOVERLAP)
        IL_004e: stfld System.Threading._IOCompletionCallback::_pOVERLAP
        IL_0053: ldloc.1 
        IL_0054: ldfld System.Threading._IOCompletionCallback::_executionContext
        IL_0059: callvirt System.Threading.ExecutionContext::CreateCopy()
        IL_005e: stloc.3 
    .try {
        IL_005f: ldloc.3 
        IL_0060: ldsfld System.Threading._IOCompletionCallback::_ccb
        IL_0065: ldloc.1 
        IL_0066: ldc.i4.1 
        IL_0067: call System.Threading.ExecutionContext::Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, object, bool)
        IL_006c: leave.s IL_0078
            72665c83 837f0800        cmp     dword ptr [edi+8],0
            72665c87 0f8527e08f00    jne     mscorlib_ni+0xb93cb4 (72f63cb4)
            72665c8d ebbb            jmp     mscorlib_ni+0x295c4a (72665c4a)
    } // END TRY (IL_005f)
    .finally {
        IL_006e: ldloc.3 
        IL_006f: brfalse.s IL_0077
            72665c8f 837dd000        cmp     dword ptr [ebp-30h],0
            72665c93 0f84d4df8f00    je      mscorlib_ni+0xb93c6d (72f63c6d)
            72665c99 e9c6df8f00      jmp     mscorlib_ni+0xb93c64 (72f63c64)
        IL_0071: ldloc.3 
        IL_0072: callvirt System.IDisposable::Dispose()
            72f63c64 8b4dd0          mov     ecx,dword ptr [ebp-30h]
            72f63c67 ff15d8524672    call    dword ptr [mscorlib_ni+0x952d8 (724652d8)]
        IL_0077: endfinally 
    } // END FINALLY (IL_006e)
        IL_0078: ldarga.s 2 (pOVERLAP)
        IL_007a: ldarga.s 0 (errorCode)
        IL_007c: ldarga.s 1 (numBytes)
        IL_007e: call System.Threading.OverlappedData::CheckVMForIOPacket(System.Threading.NativeOverlapped*&, uint&, uint&)
        IL_0083: ldarg.2  (pOVERLAP)
        IL_0084: ldc.i4.0 
        IL_0085: conv.u 
        IL_0086: bne.un IL_0000
        IL_008b: ret 
            72f63c6d 58              pop     eax
            72f63c6e ffe0            jmp     eax
            72f63c70 c745e400000000  mov     dword ptr [ebp-1Ch],0
            72f63c77 e9e81f70ff      jmp     mscorlib_ni+0x295c64 (72665c64)
    .try {
        IL_005f: ldloc.3 
        IL_0060: ldsfld System.Threading._IOCompletionCallback::_ccb
        IL_0065: ldloc.1 
        IL_0066: ldc.i4.1 
        IL_0067: call System.Threading.ExecutionContext::Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, object, bool)
        IL_006c: leave.s IL_0078
    } // END TRY (IL_005f)
    .finally {
        IL_006e: ldloc.3 
        IL_006f: brfalse.s IL_0077
        IL_0071: ldloc.3 
        IL_0072: callvirt System.IDisposable::Dispose()
        IL_0077: endfinally 
    } // END FINALLY (IL_006e)
        IL_0078: ldarga.s 2 (pOVERLAP)
        IL_007a: ldarga.s 0 (errorCode)
        IL_007c: ldarga.s 1 (numBytes)
        IL_007e: call System.Threading.OverlappedData::CheckVMForIOPacket(System.Threading.NativeOverlapped*&, uint&, uint&)
        IL_0083: ldarg.2  (pOVERLAP)
        IL_0084: ldc.i4.0 
        IL_0085: conv.u 
        IL_0086: bne.un IL_0000
        IL_008b: ret 
            72f63c7c bab3010000      mov     edx,1B3h
            72f63c81 b901000000      mov     ecx,1
            72f63c86 e8b52364ff      call    mscorlib_ni+0x1d6040 (725a6040)
            72f63c8b 8b907c010000    mov     edx,dword ptr [eax+17Ch]
            72f63c91 57              push    edi
            72f63c92 6a01            push    1
            72f63c94 8b4dd0          mov     ecx,dword ptr [ebp-30h]
            72f63c97 e878426aff      call    mscorlib_ni+0x237f14 (72607f14)  [System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean), MD=7244764c]
            72f63c9c c745e000000000  mov     dword ptr [ebp-20h],0
            72f63ca3 c745e4fc000000  mov     dword ptr [ebp-1Ch],0FCh
            72f63caa 68703cf672      push    offset mscorlib_ni+0xb93c70 (72f63c70)
            72f63caf e9db1f70ff      jmp     mscorlib_ni+0x295c8f (72665c8f)
        IL_001e: ldloc.1 
        IL_001f: ldfld System.Threading._IOCompletionCallback::_executionContext
        IL_0024: ldc.i4.1 
        IL_0025: callvirt System.Threading.ExecutionContext::IsDefaultFTContext(bool)
        IL_002a: brfalse.s IL_003e
        IL_002c: ldloc.0 
        IL_002d: callvirt System.Threading.Overlapped::get_UserCallback()
        IL_0032: stloc.2 
        IL_0033: ldloc.2 
        IL_0034: ldarg.0  (errorCode)
        IL_0035: ldarg.1  (numBytes)
        IL_0036: ldarg.2  (pOVERLAP)
        IL_0037: callvirt System.Threading.IOCompletionCallback::Invoke(uint, uint, System.Threading.NativeOverlapped*)
        IL_003c: br.s IL_0078
            72f63cb4 8b4f08          mov     ecx,dword ptr [edi+8]
            72f63cb7 ba01000000      mov     edx,1
            72f63cbc 3909            cmp     dword ptr [ecx],ecx
            72f63cbe e86d436aff      call    mscorlib_ni+0x238030 (72608030)  [System.Threading.ExecutionContext.IsDefaultFTContext(Boolean), MD=724476f8]
            72f63cc3 85c0            test    eax,eax
            72f63cc5 0f857f1f70ff    jne     mscorlib_ni+0x295c4a (72665c4a)
        IL_003e: ldloc.1 
        IL_003f: ldarg.0  (errorCode)
        IL_0040: stfld System.Threading._IOCompletionCallback::_errorCode
            72f63ccb 8b45d8          mov     eax,dword ptr [ebp-28h]
            72f63cce 89470c          mov     dword ptr [edi+0Ch],eax
        IL_0045: ldloc.1 
        IL_0046: ldarg.1  (numBytes)
        IL_0047: stfld System.Threading._IOCompletionCallback::_numBytes
            72f63cd1 8b45d4          mov     eax,dword ptr [ebp-2Ch]
            72f63cd4 894710          mov     dword ptr [edi+10h],eax
        IL_004c: ldloc.1 
        IL_004d: ldarg.2  (pOVERLAP)
        IL_004e: stfld System.Threading._IOCompletionCallback::_pOVERLAP
            72f63cd7 8b4508          mov     eax,dword ptr [ebp+8]
            72f63cda 894714          mov     dword ptr [edi+14h],eax
        IL_0053: ldloc.1 
        IL_0054: ldfld System.Threading._IOCompletionCallback::_executionContext
        IL_0059: callvirt System.Threading.ExecutionContext::CreateCopy()
        IL_005e: stloc.3 
    .try {
        IL_005f: ldloc.3 
        IL_0060: ldsfld System.Threading._IOCompletionCallback::_ccb
        IL_0065: ldloc.1 
        IL_0066: ldc.i4.1 
        IL_0067: call System.Threading.ExecutionContext::Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, object, bool)
        IL_006c: leave.s IL_0078
    } // END TRY (IL_005f)
    .finally {
        IL_006e: ldloc.3 
        IL_006f: brfalse.s IL_0077
        IL_0071: ldloc.3 
        IL_0072: callvirt System.IDisposable::Dispose()
        IL_0077: endfinally 
    } // END FINALLY (IL_006e)
        IL_0078: ldarga.s 2 (pOVERLAP)
        IL_007a: ldarga.s 0 (errorCode)
        IL_007c: ldarga.s 1 (numBytes)
        IL_007e: call System.Threading.OverlappedData::CheckVMForIOPacket(System.Threading.NativeOverlapped*&, uint&, uint&)
        IL_0083: ldarg.2  (pOVERLAP)
        IL_0084: ldc.i4.0 
        IL_0085: conv.u 
        IL_0086: bne.un IL_0000
        IL_008b: ret 
            72f63cdd 8b4f08          mov     ecx,dword ptr [edi+8]
            72f63ce0 3909            cmp     dword ptr [ecx],ecx
            72f63ce2 e859726dff      call    mscorlib_ni+0x26af40 (7263af40)  [System.Threading.ExecutionContext.CreateCopy(), MD=7244768c]
            72f63ce7 8945d0          mov     dword ptr [ebp-30h],eax
            72f63cea eb90            jmp     mscorlib_ni+0xb93c7c (72f63c7c)
    Invalid offset (630153f7) passed to ILDecoder.

    ==========

    now i want to know how this code is executed, which service is executing this and stack trace indicate that,

    14A8EC74 0B10B8BA UNKNOWN!System.Runtime.Fx+IOCompletionThunk.UnhandledExceptionFrame(UInt32, UInt32, System.Threading.NativeOverlapped*)+0x4a
        14A8ECA4 72665C64 mscorlib_ni!System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)+0x74

    so kind of System.Threading._IOCompletionCallback function is executed. now how i know who is calling this function. and how in producting we are receiving 130 mb message. how can i debug or find the root casue. can someone help me to find out the root cause?

    can someone help me how to fig out root cause or in which wcf service this problem is there. this stack trace indicate system.servicemode codebase. and i am also want to know from where this code is executed. In production, it is running multiple serivces.

    it indicate some king of IOCompletion call back function is called and unhandled exception raised while allocationg System.Runtime.Fx.AllocateByteArray(Int32) in wcf service.

    System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)(+0x78 IL,+0x74 Native)

    can someone help me how to fig out root cause or in which wcf service this problem is there. this stack trace indicate system.servicemode codebase. and i am also want to know from where this code is executed. In production, it is running multiple serivces.


    Friday, April 26, 2013 10:58 AM

Answers

  • This definitely looks like the program that caused the exception, but may not be.  The interesting part of the dump is the following message : "Address 0 not found in the managed heap".  A heap as a signature and the beginning of the object (memory location 0) and it appears that the signature got corrupted.  Your problem may be a "out of memory" issue or it could be data that is being written to the wrong location.  It turns out the most common data value in any program is zero.  So often if a pointer (which get initialized to zero) isn't set properly it willoften  corrupt memory location zero.  A heap object also contains size information at the beginning of the heap and it is possible that you are just creating a very large heap.

    jdweng

    Monday, April 29, 2013 7:07 AM

All replies

  • I suspect that once you start to run out of memeory more than one process/service starts to fail and you are getting more than one process giving an out of memory exception.  I would watch the task manager and sort the process by memory size to find out which  executable is hogging the memory.  You know how to repliocate the problem so I would debug the method you are using to find the problem.

    jdweng

    Friday, April 26, 2013 12:17 PM
  • there are two servers and this is happening on one server. on that server in last 4 month this happend only 5 times.  so time frame is not exact. event i am not knowing the pattern. 

    from dump, i got below infomation :

    0:052> !address -summary

                                         
    Failed to map Heaps (error 80004005)

    --- Usage Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal
    <unclassified>                         2999          48e6c000 (   1.139 Gb)  85.70%   56.96%
    Free                                    416          2aee2000 ( 686.883 Mb)           33.54%
    Image                                  1116           a5f6000 ( 165.961 Mb)  12.19%    8.10%
    Stack                                   186           1c34000 (  28.203 Mb)   2.07%    1.38%
    TEB                                      62             3e000 ( 248.000 kb)   0.02%    0.01%
    NlsTables                                 1             23000 ( 140.000 kb)   0.01%    0.01%
    ActivationContextData                    10             11000 (  68.000 kb)   0.00%    0.00%
    CsrSharedMemory                           1              5000 (  20.000 kb)   0.00%    0.00%
    PEB                                       1              1000 (   4.000 kb)   0.00%    0.00%

    --- Type Summary (for busy) ------ RgnCount ----------- Total Size -------- %ofBusy %ofTotal
    MEM_PRIVATE                            2530          46e1a000 (   1.108 Gb)  83.33%   55.38%
    MEM_IMAGE                              1728           d1fc000 ( 209.984 Mb)  15.43%   10.25%
    MEM_MAPPED                              118           10f8000 (  16.969 Mb)   1.25%    0.83%

    --- State Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal
    MEM_COMMIT                             3405          47d66000 (   1.122 Gb)  84.45%   56.12%
    MEM_FREE                                416          2aee2000 ( 686.883 Mb)           33.54%
    MEM_RESERVE                             971           d3a8000 ( 211.656 Mb)  15.55%   10.34%

    --- Protect Summary (for commit) - RgnCount ----------- Total Size -------- %ofBusy %ofTotal
    PAGE_READWRITE                         1349          37caf000 ( 892.684 Mb)  65.59%   43.59%
    PAGE_EXECUTE_READ                       240           a6d7000 ( 166.840 Mb)  12.26%    8.15%
    PAGE_EXECUTE_READWRITE                  773           27aa000 (  39.664 Mb)   2.91%    1.94%
    PAGE_READONLY                           623           2317000 (  35.090 Mb)   2.58%    1.71%
    PAGE_WRITECOPY                          209            c50000 (  12.313 Mb)   0.90%    0.60%
    PAGE_EXECUTE_WRITECOPY                   58            177000 (   1.465 Mb)   0.11%    0.07%
    PAGE_READWRITE|PAGE_GUARD               153            158000 (   1.344 Mb)   0.10%    0.07%

    --- Largest Region by Usage ----------- Base Address -------- Region Size ----------
    <unclassified>                              41290000           8002000 ( 128.008 Mb)
    Free                                        775a0000           78cd000 ( 120.801 Mb)
    Image                                       6c4b0000            dc2000 (  13.758 Mb)
    Stack                                        34b0000             fd000 (1012.000 kb)
    TEB                                         7ee6f000              1000 (   4.000 kb)
    NlsTables                                   7efb0000             23000 ( 140.000 kb)
    ActivationContextData                          50000              4000 (  16.000 kb)
    CsrSharedMemory                             7efe0000              5000 (  20.000 kb)
    PEB                                         7efde000              1000 (   4.000 kb)

    if you need any more info let me know.

    Friday, April 26, 2013 12:30 PM
  • Maybe checking the Event Viewer will give more information around the time of the exception. 
    Can you tell me where the data is going and why you need ot keep it in memory. 

    Normally a service will have two ports (input and output) and the service routine should simply accept data on the input port and send data on the output port.   You should use AsyncRead (Non Blocking) and/or AsyncWrite (Non blocking).  Do use Read and/or Write which are blocking and won't release memory until the entire data is received/transmitted.


    jdweng

    Friday, April 26, 2013 1:04 PM
  • i have no idea..but inside we have written code which continuously do polling and collect "countrs" data like no of message received, processed etc, for user services. this exception is coming while calling close () method of proxy. but my question is how i start to dig more in windbg. bcz stack trace is limited. how i know what that message bcz we are not using that much big size of message in our service. we are not uploading or sending any files.

    need to know how i know which service is calling or from where request comes and message contet.

    can you help me to fig. out this using windbg. not able to find any other way to get/dig more on this. in production environment..the issue is reproduced but patter and time frame is not fix.

    Friday, April 26, 2013 1:13 PM
  • A proxy service is using port forwarding which means the data coming in on one port number and then going out on a different port number.  If the proxy server stops processing the data then the data could get backed up on your server and cause a out of memory exception.  Sometimes the proxy server has limitations on the size of data that can be sent.  I suspect the proxy server is closing the connection which is the root cause of the problem.   You need to add an event handler into your service that checks if the connection is closed and release any memory (objects) that your server is using.

    jdweng

    Friday, April 26, 2013 1:27 PM
  • currently not knowing which service is throwing exception. there are around 9 core services. only authentication, authorization and service registry using wshttp binding. as i know it will buffer whole message befor encrypting/decrypting. i have only dump and i do not know how to find all this info from this.

    could you guid me how to proceed further? is it possible to find out from where request is coming and reply going. let me know if you have any suggessation.

    Friday, April 26, 2013 5:38 PM
  • These type problems are always tough to find.  Usually adding additional debug messages into log file always help.  I still curious why you are saying that encryption/decryption buffer entire data.  This is not necessary.  Encryption algorithms work on buffer of less than 256 bytes and only have to store one buffer of data.  With http 1.0 you can use chunk mode so you don't have to buffer entire message.  If you know that certain services are buffering large amount of data then just fix them which may take laess time then actually tracing which is cuasing the problem.

    From my experiences memory exception usually occur because a connection closes unexpectively.  With TCP when the receiving end of a connection closes it often causes memory exception on the sending end.  The send end has to buffer all the data until an ack occurs.  Also every packet ends up getting resent 5 times which loads down the ethernet interface.  Sometimes using the command "netstat -a" and checking for ports that are 1/2 closed just after the exception occurs may give a clue.


    jdweng

    Friday, April 26, 2013 7:37 PM
  • as i know in config file for wcf servie we are using wshttp binding , see below section :

    <wsHttpBinding>
            <binding name="WSHttpBinding_Authentication" closeTimeout="00:01:00" openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:01:00" bypassProxyOnLocal="false" transactionFlow="false" hostNameComparisonMode="StrongWildcard" maxBufferPoolSize="2147483647" maxReceivedMessageSize="2147483647" messageEncoding="Text" textEncoding="utf-8" useDefaultWebProxy="true" allowCookies="false">
            </binding>

    <wsHttpBinding>

    =================

           <binding name="WSHttpBinding_NotificationRegistry" closeTimeout="00:01:00" openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:01:00" bypassProxyOnLocal="false" transactionFlow="false" hostNameComparisonMode="StrongWildcard" maxBufferPoolSize="2147483647" maxReceivedMessageSize="2147483647" messageEncoding="Text" textEncoding="utf-8" useDefaultWebProxy="true" allowCookies="false">
              <readerQuotas maxDepth="32" maxStringContentLength="2147483647" maxArrayLength="2147483647" maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647" />
               <readerQuotas maxDepth="32" maxStringContentLength="2147483647" maxArrayLength="2147483647" maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647" />
              <reliableSession ordered="true" inactivityTimeout="00:10:00" enabled="false" />
              <security mode="Message">
                <message clientCredentialType="Windows" />
              </security>     </Binding>

          </wsHttpBinding>

    so as i know if we use wshttp binding and security mode is "message" then in that case when it receives request/reply it will buffer the whole message & try to do serialization/deserialization the whole soap message. that's why i believe, that service receive somehow 130 mb soap message and during closing() proxy it try to copy to buffer and during that time available buffer size is 120 mb and message size is 136 mb. this size i found from dump. 

    this is my understanding but if you could help me technically then it would be helpful.Total 8 services and running separate app domain. let  me know if u need any more info. i have provided memory info from dump, could you also help me to interpret what it indicate.



    Saturday, April 27, 2013 5:03 AM
  • thanks for giving suggessation. i am kwowing if we use large files then we have to used streaming. but as i mention we are not using any big files in service and currently we are not knowing for which service we have to change.

    i just found below xml message from windbg..it's size is around 15kb. not knowing how it is generated.

    ===========

    0:041> !do 01fab8c8

    Name:        System.String

    MethodTable: 726efb08

    EEClass:     72428bb0

    Size:        9518(0x252e) bytes

    File:        C:\Windows\Microsoft.Net\assembly\GAC_32\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll

    String:      <?xml version='1.0' encoding='UTF-8' ?>

    <xs:schema xmlns:soap='http://schemas.xmlsoap.org/wsdl/soap/' xmlns:wsdl='http://schemas.xmlsoap.org/wsdl/' targetNamespace='http://schemas.xmlsoap.org/wsdl/soap/' xmlns:xs='http://www.w3.org/2001/XMLSchema'>

      <xs:import namespace='http://schemas.xmlsoap.org/wsdl/' />

      <xs:simpleType name='encodingStyle'>

        <xs:annotation>

          <xs:documentation>

          'encodingStyle' indicates any canonicalization conventions followed in the contents of the containing element.  For example, the value 'http://schemas.xmlsoap.org/soap/encoding/' indicates the pattern described in SOAP specification

          </xs:documentation>

        </xs:annotation>

        <xs:list itemType='xs:anyURI' />

      </xs:simpleType>

      <xs:element name='binding' type='soap:tBinding' />

      <xs:complexType name='tBinding'>

        <xs:complexContent mixed='false'>

          <xs:extension base='wsdl:tExtensibilityElement'>

            <xs:attribute name='transport' type='xs:anyURI' use='required' />

            <xs:attribute name='style' type='soap:tStyleChoice' use='optional' />

          </xs:extension>

        </xs:complexContent>

      </xs:complexType>

      <xs:simpleType name='tStyleChoice'>

        <xs:restriction base='xs:string'>

          <xs:enumeration value='rpc' />

          <xs:enumeration value='document' />

        </xs:restriction>

      </xs:simpleType>

      <xs:element name='operation' type='soap:tOperation' />

      <xs:complexType name='tOperation'>

        <xs:complexContent mixed='false'>

          <xs:extension base='wsdl:tExtensibilityElement'>

            <xs:attribute name='soapAction' type='xs:anyURI' use='optional' />

            <xs:attribute name='style' type='soap:tStyleChoice' use='optional' />

          </xs:extension>

        </xs:complexContent>

      </xs:complexType>

      <xs:element name='body' type='soap:tBody' />

      <xs:attributeGroup name='tBodyAttributes'>

        <xs:attribute name='encodingStyle' type='soap:encodingStyle' use='optional' />

        <xs:attribute name='use' type='soap:useChoice' use='optional' />

        <xs:attribute name='namespace' type='xs:anyURI' use='optional' />

      </xs:attributeGroup>

      <xs:complexType name='tBody'>

        <xs:complexContent mixed='false'>

          <xs:extension base='wsdl:tExtensibilityElement'>

            <xs:attribute name='parts' type='xs:NMTOKENS' use='optional' />

            <xs:attributeGroup ref='soap:tBodyAttributes' />

          </xs:extension>

        </xs:complexContent>

      </xs:complexType>

      <xs:simpleType name='useChoice'>

        <xs:restriction base='xs:string'>

          <xs:enumeration value='literal' />

          <xs:enumeration value='encoded' />

        </xs:restriction>

      </xs:simpleType>

      <xs:element name='fault' type='soap:tFault' />

      <xs:complexType name='tFaultRes' abstract='true'>

        <xs:complexContent mixed='false'>

          <xs:restriction base='soap:tBody'>

            <xs:attribute ref='wsdl:required' use='optional' />

            <xs:attribute name='parts' type='xs:NMTOKENS' use='prohibited' />

            <xs:attributeGroup ref='soap:tBodyAttributes' />

          </xs:restriction>

        </xs:complexContent>

      </xs:complexType>

      <xs:complexType name='tFault'>

        <xs:complexContent mixed='false'>

          <xs:extension base='soap:tFaultRes'>

            <xs:attribute name='name' type='xs:NCName' use='required' />

          </xs:extension>

        </xs:complexContent>

      </xs:complexType>

      <xs:element name='header' type='soap:tHeader' />

      <xs:attributeGroup name='tHeaderAttributes'>

        <xs:attribute name='message' type='xs:QName' use='required' />

        <xs:attribute name='part' type='xs:NMTOKEN' use='required' />

        <xs:attribute name='use' type='soap:useChoice' use='required' />

        <xs:attribute name='encodingStyle' type='soap:encodingStyle' use='optional' />

        <xs:attribute name='namespace' type='xs:anyURI' use='optional' />

      </xs:attributeGroup>

      <xs:complexType name='tHeader'>

        <xs:complexContent mixed='false'>

          <xs:extension base='wsdl:tExtensibilityElement'>

            <xs:sequence>

              <xs:element minOccurs='0' maxOccurs='unbounded' ref='soap:headerfault' />

            </xs:sequence>

            <xs:attributeGroup ref='soap:tHeaderAttributes' />

          </xs:extension>

        </xs:complexContent>

      </xs:complexType>

      <xs:element name='headerfault' type='soap:tHeaderFault' />

      <xs:complexType name='tHeaderFault'>

        <xs:attributeGroup ref='soap:tHeaderAttributes' />

      </xs:complexType>

      <xs:element name='address' type='soap:tAddress' />

      <xs:complexType name='tAddress'>

        <xs:complexContent mixed='false'>

          <xs:extension base='wsdl:tExtensibilityElement'>

            <xs:attribute name='location' type='xs:anyURI' use='required' />

          </xs:extension>

        </xs:complexContent>

      </xs:complexType>

    </xs:schema>

    Fields:

          MT    Field   Offset                 Type VT     Attr    Value Name

    726f2ad4  4000103        4         System.Int32  1 instance     4752 m_stringLength

    726f1f24  4000104        8          System.Char  1 instance       3c m_firstChar

    726efb08  4000105        8        System.String  0   shared   static Empty

        >> Domain:Value  00434028:01001228 07e0c8a0:01001228 0905df50:01001228 0918cc00:01001228 091e5d68:01001228 092317e0:01001228 09284948:01001228 092a9bb8:01001228 092d8a30:01001228 09300718:01001228 0930ca80:01001228 09338a80:01001228 09358d18:01001228 0937f540:01001228 0d505d98:01001228 <<

    0:041> !gcroot 01fab8c8  

    Note: Roots found on stacks may be false positives. Run "!help gcroot" for

    more info.

    Scan Thread 0 OSTHread 4510

    Scan Thread 2 OSTHread 1664

    Scan Thread 4 OSTHread 35d4

    Scan Thread 5 OSTHread 2ffc

    Scan Thread 26 OSTHread 2690

    Scan Thread 28 OSTHread 65b0

    Scan Thread 29 OSTHread 7af0

    Scan Thread 30 OSTHread 9fb8

    Scan Thread 31 OSTHread 7c50

    Scan Thread 33 OSTHread 42c4

    Scan Thread 36 OSTHread 3a4c

    Scan Thread 6 OSTHread 1a3c

    Scan Thread 37 OSTHread 52bc

    Scan Thread 38 OSTHread a22c

    Scan Thread 39 OSTHread 9fc

    Scan Thread 40 OSTHread 64a4

    Scan Thread 41 OSTHread 255c

    Scan Thread 42 OSTHread 8d14

    Scan Thread 43 OSTHread 2818

    Scan Thread 44 OSTHread 6484

    Scan Thread 45 OSTHread 528

    Scan Thread 46 OSTHread 3c3c

    Scan Thread 47 OSTHread a744

    Scan Thread 48 OSTHread 4fcc

    Scan Thread 49 OSTHread 75e0

    Scan Thread 50 OSTHread 6564

    Scan Thread 51 OSTHread 81d8

    Scan Thread 52 OSTHread 5f4c

    Scan Thread 53 OSTHread 3748

    Scan Thread 54 OSTHread 328c

    Scan Thread 55 OSTHread a850

    Scan Thread 56 OSTHread 301c

    Scan Thread 57 OSTHread bafc

    DOMAIN(00434028):HANDLE(Pinned):1013b8:Root:  0205d650(System.Object[])->

      01fab8c8(System.String)

    ======

    let me know if you have any information on this and how this xml is generated.

    Saturday, April 27, 2013 11:13 AM
  • This may be a false lead.  When you start to run out of memory the process that is using the memory may not be the one to cause the exception.  When you are running more than one process at a time all the processes are sharing the same memory so when memory is low the process that gives the exception may not be the one that has used all the memory.  You have to find the process that is using all the memory.  Looking at the task manager may help.

    Try using the performance monitor.  See webpage below

    http://stackoverflow.com/questions/69332/tracking-cpu-and-memory-usage-per-process


    jdweng

    Saturday, April 27, 2013 11:49 AM
  • found below things from dump :

    0:052> !GCWhere
    Address 0 not found in the managed heap.
    0:052> !HeapStat
    Heap             Gen0         Gen1         Gen2          LOH
    Heap0         7891104       209236     64798764    425101368

    Free space:                                                 Percentage
    Heap0         6941680         1692     25898056     36942056SOH: 45% LOH:  8%
    0:052> !analyzeOOM
    No export analyzeOOM found
    0:052> !AnalyzeOOM
    Managed OOM occured after GC #425381 (Requested to allocate 134217744 bytes)
    Reason: Didn't have enough memory to allocate an LOH segment
    Detail: LOH: Failed to reserve memory (142606336 bytes)


    Monday, April 29, 2013 4:39 AM
  • This definitely looks like the program that caused the exception, but may not be.  The interesting part of the dump is the following message : "Address 0 not found in the managed heap".  A heap as a signature and the beginning of the object (memory location 0) and it appears that the signature got corrupted.  Your problem may be a "out of memory" issue or it could be data that is being written to the wrong location.  It turns out the most common data value in any program is zero.  So often if a pointer (which get initialized to zero) isn't set properly it willoften  corrupt memory location zero.  A heap object also contains size information at the beginning of the heap and it is possible that you are just creating a very large heap.

    jdweng

    Monday, April 29, 2013 7:07 AM
  • i found if we have to use !GCWhere then we have to prived memory address. i havn't given here so it it giving message like "Address 0 not found in the managed heap.".

    Friday, May 3, 2013 11:12 AM
  • Do you check if GCWhere = null before !GCWhere?

    jdweng

    Friday, May 3, 2013 11:59 AM