Hanging in MFC using VS2008sp1 on Windows 7

Question

Hello,

I am experiencing a hang in MFC code while running VS2008sp1 (with all updates) on Windows 7 (RTM will all updates).
This does not happen on XP, nor with VS2005 (with all updates).

The code is in the following file and function:

File:      dlgfile.cpp
Function:  INT_PTR CFileDialog::DoModal()

VS2005sp1
    if (m_ofn.Flags & OFN_EXPLORER)
        pThreadState->m_pAlternateWndInit = this;
    else
        AfxHookWindowCreate(this);

    INT_PTR nResult;
    if (m_bOpenFileDialog)
        nResult = ::AfxCtxGetOpenFileName(&m_ofn);
    else
        nResult = ::AfxCtxGetSaveFileName(&m_ofn);


VS2008sp1 (hangs in Windows 7 but not in XP)

    if (m_bVistaStyle == TRUE)
    {
        AfxHookWindowCreate(this);
    }
    else if (m_ofn.Flags & OFN_EXPLORER)
        pThreadState->m_pAlternateWndInit = this;
    else
        AfxHookWindowCreate(this);

    INT_PTR nResult = 0;

    if (m_bVistaStyle == TRUE)
    {
        ApplyOFNToShellDialog();
        HRESULT hr = (static_cast<IFileDialog*>(m_pIFileDialog))->Show(m_ofn.hwndOwner); // hangs here in Windows 7
        nResult = (hr == S_OK) ? IDOK : IDCANCEL;
    }
    else if (m_bOpenFileDialog)
        nResult = ::AfxCtxGetOpenFileName(&m_ofn);
    else
        nResult = ::AfxCtxGetSaveFileName(&m_ofn);


The new code in VS2008 adds an if clause, checking the value of m_bVistaStyle.  This clause is entered while running on Windows 7, but not XP.  I have not tried it on Vista.

Please provide me any guidance you can on this matter, as we are unable to debug passed this issue.

Thank you, in advance,

-- Rich

Answer 1

Use SysInternals' AutoRuns utility and disable all the non-Microsoft shell extension handlers.  Then try again.  If it still hangs, make sure you got the Windows debugging symbols, use Debug + Break All when it hangs and post the stack trace of the UI thread.

---

Hans Passant.

Answer 2

Here is the call stack...


    user32.dll!_NtUserWaitMessage@0()  + 0x15 bytes   
     user32.dll!_NtUserWaitMessage@0()  + 0x15 bytes   
     user32.dll!_DialogBox2@16()  + 0x109 bytes   
     user32.dll!_InternalDialogBox@24()  + 0xc9 bytes   
     user32.dll!_DialogBoxIndirectParamAorW@24()  + 0x36 bytes   
     user32.dll!_DialogBoxIndirectParamW@20()  + 0x1b bytes   
     comdlg32.dll!CFileOpenSave::Show()  + 0x146 bytes   
>    mfc90ud.dll!CFileDialog::DoModal()  Line 706 + 0x26 bytes    C++
     TracePro70X_D.exe!CTraceProApp::DoPromptFileName(ATL::CStringT<wchar_t,StrTraitMFC_DLL<wchar_t,ATL::ChTraitsCRT<wchar_t> > > & fileName="", unsigned int nIDSTitle=61440, unsigned long lFlags=4100, int bOpenFileDialog=1, CDocTemplate * pTemplate=0x00000000)  Line 2783 + 0xb bytes    C++
     TracePro70X_D.exe!CTraceProApp::OnFileOpen()  Line 2587 + 0x22 bytes    C++
     mfc90ud.dll!_AfxDispatchCmdMsg(CCmdTarget * pTarget=0x011fcc78, unsigned int nID=57601, int nCode=0, void (void)* pfn=0x00403d96, void * pExtra=0x00000000, unsigned int nSig=57, AFX_CMDHANDLERINFO * pHandlerInfo=0x00000000)  Line 82    C++
     mfc90ud.dll!CCmdTarget::OnCmdMsg(unsigned int nID=57601, int nCode=0, void * pExtra=0x00000000, AFX_CMDHANDLERINFO * pHandlerInfo=0x00000000)  Line 381 + 0x27 bytes    C++
     mfc90ud.dll!CFrameWnd::OnCmdMsg(unsigned int nID=57601, int nCode=0, void * pExtra=0x00000000, AFX_CMDHANDLERINFO * pHandlerInfo=0x00000000)  Line 951 + 0x23 bytes    C++
     mfc90ud.dll!CWnd::OnCommand(unsigned int wParam=57601, long lParam=395304)  Line 2364    C++
     mfc90ud.dll!CFrameWnd::OnCommand(unsigned int wParam=57601, long lParam=395304)  Line 366    C++
     mfc90ud.dll!CWnd::OnWndMsg(unsigned int message=273, unsigned int wParam=57601, long lParam=395304, long * pResult=0x0018f308)  Line 1769 + 0x1e bytes    C++
     mfc90ud.dll!CWnd::WindowProc(unsigned int message=273, unsigned int wParam=57601, long lParam=395304)  Line 1755 + 0x20 bytes    C++
     mfc90ud.dll!AfxCallWndProc(CWnd * pWnd=0x08766fa0, HWND__ * hWnd=0x000b07be, unsigned int nMsg=273, unsigned int wParam=57601, long lParam=395304)  Line 240 + 0x1c bytes    C++
     mfc90ud.dll!CMDIFrameWnd::OnCommand(unsigned int wParam=57601, long lParam=395304)  Line 48 + 0x23 bytes    C++
     mfc90ud.dll!CWnd::OnWndMsg(unsigned int message=273, unsigned int wParam=57601, long lParam=395304, long * pResult=0x0018f51c)  Line 1769 + 0x1e bytes    C++
     mfc90ud.dll!CWnd::WindowProc(unsigned int message=273, unsigned int wParam=57601, long lParam=395304)  Line 1755 + 0x20 bytes    C++
     mfc90ud.dll!AfxCallWndProc(CWnd * pWnd=0x0875ecf0, HWND__ * hWnd=0x00070784, unsigned int nMsg=273, unsigned int wParam=57601, long lParam=395304)  Line 240 + 0x1c bytes    C++
     mfc90ud.dll!AfxWndProc(HWND__ * hWnd=0x00070784, unsigned int nMsg=273, unsigned int wParam=57601, long lParam=395304)  Line 403    C++
     mfc90ud.dll!AfxWndProcBase(HWND__ * hWnd=0x00070784, unsigned int nMsg=273, unsigned int wParam=57601, long lParam=395304)  Line 441 + 0x15 bytes    C++
     user32.dll!_InternalCallWinProc@20()  + 0x23 bytes   
     user32.dll!_UserCallWinProcCheckWow@32()  + 0xb7 bytes   
     user32.dll!_DispatchClientMessage@24()  + 0x51 bytes   
     user32.dll!___fnDWORD@4()  + 0x2b bytes   
     ntdll.dll!_KiUserCallbackDispatcher@12()  + 0x2e bytes   
     user32.dll!_NtUserMessageCall@28()  + 0x15 bytes   
     user32.dll!_SendMessageWorker@24()  + 0x4ff9 bytes   
     user32.dll!_SendMessageW@16()  + 0x4c bytes   
     comctl32.dll!_TBOnLButtonUp@20()  + 0x131 bytes   
     comctl32.dll!_ToolbarWndProc@16()  + 0xba8 bytes   
     user32.dll!_InternalCallWinProc@20()  + 0x23 bytes   
     user32.dll!_UserCallWinProcCheckWow@32()  + 0xb7 bytes   
     user32.dll!_CallWindowProcAorW@24()  + 0x5e bytes   
     user32.dll!_CallWindowProcW@20()  + 0x1b bytes   
     mfc90ud.dll!CWnd::DefWindowProcW(unsigned int nMsg=514, unsigned int wParam=0, long lParam=851999)  Line 1043 + 0x20 bytes    C++
     mfc90ud.dll!CWnd::WindowProc(unsigned int message=514, unsigned int wParam=0, long lParam=851999)  Line 1756 + 0x1c bytes    C++
     mfc90ud.dll!CControlBar::WindowProc(unsigned int nMsg=514, unsigned int wParam=0, long lParam=851999)  Line 506 + 0x14 bytes    C++
     mfc90ud.dll!AfxCallWndProc(CWnd * pWnd=0x0875f504, HWND__ * hWnd=0x00060828, unsigned int nMsg=514, unsigned int wParam=0, long lParam=851999)  Line 240 + 0x1c bytes    C++
     mfc90ud.dll!AfxWndProc(HWND__ * hWnd=0x00060828, unsigned int nMsg=514, unsigned int wParam=0, long lParam=851999)  Line 403    C++
     mfc90ud.dll!AfxWndProcBase(HWND__ * hWnd=0x00060828, unsigned int nMsg=514, unsigned int wParam=0, long lParam=851999)  Line 441 + 0x15 bytes    C++
     user32.dll!_InternalCallWinProc@20()  + 0x23 bytes   
     user32.dll!_UserCallWinProcCheckWow@32()  + 0xb7 bytes   
     user32.dll!_DispatchMessageWorker@8()  + 0xed bytes   
     user32.dll!_DispatchMessageW@4()  + 0xf bytes   
     user32.dll!_IsDialogMessageW@8()  + 0x11e bytes   
     mfc90ud.dll!CWnd::IsDialogMessageW(tagMSG * lpMsg=0x0145d118)  Line 198    C++
     mfc90ud.dll!CWnd::PreTranslateInput(tagMSG * lpMsg=0x0145d118)  Line 4332    C++
     mfc90ud.dll!CControlBar::PreTranslateMessage(tagMSG * pMsg=0x0145d118)  Line 445 + 0xc bytes    C++
     mfc90ud.dll!CWnd::WalkPreTranslateTree(HWND__ * hWndStop=0x00070784, tagMSG * pMsg=0x0145d118)  Line 2946 + 0x14 bytes    C++
     mfc90ud.dll!AfxInternalPreTranslateMessage(tagMSG * pMsg=0x0145d118)  Line 233 + 0x12 bytes    C++
     mfc90ud.dll!CWinThread::PreTranslateMessage(tagMSG * pMsg=0x0145d118)  Line 777 + 0x9 bytes    C++
     TracePro70X_D.exe!CTraceProApp::PreTranslateMessage(tagMSG * pMsg=0x0145d118)  Line 2334    C++
     mfc90ud.dll!AfxPreTranslateMessage(tagMSG * pMsg=0x0145d118)  Line 252 + 0x11 bytes    C++
     mfc90ud.dll!AfxInternalPumpMessage()  Line 178 + 0x18 bytes    C++
     mfc90ud.dll!CWinThread::PumpMessage()  Line 900    C++
     mfc90ud.dll!CWinThread::Run()  Line 629 + 0xd bytes    C++
     mfc90ud.dll!CWinApp::Run()  Line 865    C++
     mfc90ud.dll!AfxWinMain(HINSTANCE__ * hInstance=0x00400000, HINSTANCE__ * hPrevInstance=0x00000000, wchar_t * lpCmdLine=0x01443148, int nCmdShow=1)  Line 47 + 0xd bytes    C++
     TracePro70X_D.exe!wWinMain(HINSTANCE__ * hInstance=0x00400000, HINSTANCE__ * hPrevInstance=0x00000000, wchar_t * lpCmdLine=0x01443148, int nCmdShow=1)  Line 34    C++
     TracePro70X_D.exe!__tmainCRTStartup()  Line 578 + 0x35 bytes    C
     TracePro70X_D.exe!wWinMainCRTStartup()  Line 403    C
     kernel32.dll!@BaseThreadInitThunk@12()  + 0x12 bytes   
     ntdll.dll!___RtlUserThreadStart@8()  + 0x27 bytes   
     ntdll.dll!__RtlUserThreadStart@8()  + 0x1b bytes   

Answer 3

Do you have any insight into this?  I look forward to your response.

Thank you,

-- Rich

Answer 4

I didn't get an alert.  Your call stack looks completely normal.  You clicked a toolbar button and displayed a dialog.  It is waiting for a message from Windows.  No sign of anything being hung, your dialog should paint properly if you move another window across it.

---

Hans Passant.

Answer 5

Rich didn't say he was getting an alert, he said he was getting a hang. More specifically it is an unhandled exception error: Mine is Unhandled exception at 0x746ce898 in Project.exe: 0xC000005: Access violation reading location 0x00000024. If you break at the "if (m_bVistaStyle...." and coninue from "else if (..." everything works fine. if (m_bVistaStyle == TRUE) { ApplyOFNToShellDialog(); HRESULT hr = (static_cast<IFileDialog*>(m_pIFileDialog))->Show(m_ofn.hwndOwner); nResult = (hr == S_OK) ? IDOK : IDCANCEL; } else if (m_bOpenFileDialog) nResult = ::AfxCtxGetOpenFileName(&m_ofn); else nResult = ::AfxCtxGetSaveFileName(&m_ofn); Perhaps you could ask the guy who wrote: "HRESULT hr = (static_cast<IFileDialog*>(m_pIFileDialog))->Show(m_ofn.hwndOwner);" for a work around because this is causing a hard crash of my app which worked fine under XP. Thanks, Al

Answer 6

Arliacne,

i am facing the same issue as you stated above. Its a bloody crash when i open a dialog box and for some reasons it doesn't happen always. could you tell me how you solved this if you have solved or any place to look for  to resolve this issue.

Answer 7

i too got the same problem. i dont get any exception in ::AfxCtxGetOpenFileName(&m_ofn) on Windows XP, but in Windows 7, it fails. does anyone have solution?

Answer 8

I know the thread is old, but does anyone have any update to share?

Any new information? Or work around?

Thanks

---

Cameron Conacher