locked
.net Web Application Hangs When Finalizer Gets Blocked RRS feed

  • Question

  • We have one application runnnig for more a year, and and since 2 months ago it becomes to hang. In the meanwhile we're able to get two dumps of the hung process, and te results were analyzed by debugdiag, telling us that we must suspect on finalizer thread blocked, due to the amount of object on finalizer queue.

    Starting from information provided by debugdiag, and by mean of windbg, we've checked the following cirumstances at the moment of chrash: - All threads are on Preemptive mode - Finalizer's stack shows ntdll!NtWaitForMultipleObjects+a or ntdll!NtWaitForSingleObject+a

    Application is using managed code, but on "Previous .NET Exceptions Report (Exceptions in all .NET Heaps)" section of debugdiag result, we can see System.ExecutionEngineException out of another exceptions tha are suspicious. For example System.OutOfMemoryException and System.StackOverflowException. These only appear once, but could be related to blocked finalizer (or not??)

    Some data about environment

    • CLR Version 4.0.30319.36440
    • Operating System Windows Server 2012 R2
    • Number Of Processors 4
    • Process Bitness 64-Bit
    • Processor Type X64

    Finalizer thread stack looks like this:

    ntdll!NtWaitForSingleObject+0xa
    KERNELBASE!WaitForSingleObjectEx+0x94
    clr!CLREventWaitHelper2+0x38
    clr!CLREventWaitHelper+0x1f
    clr!CLREventBase::WaitEx+0x63
    clr!SVR::WaitForFinalizerEvent+0x4e
    clr!SVR::GCHeap::FinalizerThreadWorker+0x4a
    clr!ManagedThreadBase_DispatchInner+0x2d
    clr!ManagedThreadBase_DispatchMiddle+0x6c
    clr!ManagedThreadBase_DispatchOuter+0x75
    clr!SVR::GCHeap::FinalizerThreadStart+0xd7
    clr!Thread::intermediateThreadProc+0x7d
    kernel32!BaseThreadInitThunk+0x22
    ntdll!RtlUserThreadStart+0x34

    We've spent a lot of time trying to figure out what is happening here, but with no success, we followed several related post too, everything was unuseful.

    Could someone give us a hint on this issue?

    Many thanks in advance

    Monday, August 6, 2018 11:33 AM

All replies

  • Hi Juan M. Ríos,

    Thank you for posting here.

    For your question, what is the tool you used to debug the dump file? Debugdiag or windbg? In your description, the application is runnnig for more a year, since 2 months ago it becomes to hang. Have you do any change for this application or some updates for this PC?

    According to the Finalizer thread stack, I could not make sure what cause the exception. Could you analyze the dump file to which code hang the application? If not, could you provide the dump file for us to test?

    Best Regards,

    Wendy


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Wednesday, August 8, 2018 5:47 AM
  • Hi Wendy, thanks for your early response.

    Dump file was generated from Task Manager (OS tool), we faced some restrictions accessing production environment on which application is hosted. Therefore we asked client to use any standard OS tool to get it.

    Regarding application unstability, it was running without problems from over a year (or even more), up to now. Some information was provided recently by client about server update status, and we realized that there were installed some update pack on Jun-2th. The incident timeline is:

    - first incident was on Jun-29th, after restarting services things went right, 
    - 6 days after same incident appeared again, services were restarted - Dump file is available
    - 27 days after  it shows up again - Dump file is available

    I was using windbg trying to figure out what happens on finalizer execution stack, but without success due to my limited background on this subject, I think. 

    Surely I will provide you all information available (server update status info, two crash dump files, debugdiag result). Please send me a mail address to my personal address (juanmanuelrios@gmail.com) in order to provide a dropbox link for download these files. If you wish to use OneDrive instead of dropbox, or some other online repo please let me know, because I can't post them on this site (it returns http 500 status error), sorry.

    Thanks a lot for your support.

    Thursday, August 9, 2018 12:18 PM
  • Hi Juan M. Ríos,

    According to our policy, we could not provide any personal contact. If it is possible, you could upload your dump file on OneDrive for me to download.

    https://onedrive.live.com/

    If you want to analyze using debugdiag, you could post a new thread in Visual Studio Diagnostics (Debugger, Profiler, IntelliTrace) forum for suitable support.

    If your could not provide the dump file for security, you could use !teb and !error to get the error code. You could provide for us.

    For more details about how to get the last error for the application, please do the steps in the blog below.

    https://blogs.msdn.microsoft.com/marcelolr/2010/04/22/getlasterror-on-windbg/

    Best Regards,

    Wendy


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Friday, August 10, 2018 8:28 AM
  • Hi Wendy, thanks again for your early response.

    On this repository you'll find all the information I have about this incident: https://1drv.ms/f/s!Aimt_HGEYus8evlJP4zv9SOQZuA

    I'll try to find out what is happening inside .NET CLR at the moment of incident by my own, but I will appreciate further information you can provide on it.

    I'll take a look to the forum you pointed out, trying to find some useful information also.

    Best regards

    Friday, August 10, 2018 3:12 PM