DoDragDrop: what is best way to release mouse capture when interrupted by a modal dialog? RRS feed

  • Question

  • background: Windows 7, native desktop development using C++, working drag/drop implementation using ::DoDragDrop API (and friends).

    Recently noticed that if a modal dialog is presented during a drag operation, then the GUI becomes unresponsive. I'm quite sure this is because ::DoDragDrop captures the mouse and handles all messages until it returns. If during that time a modal dialog is presented on the UI thread, then the mouse capture still seems to be in effect such that it is impossible to click on the modal dialog or anything else (since the modal dialog's message handling seems to prevent any other messages to be handled).

    A simple (silly but reliable) way to reproduce the problem is to call ::MessageBox inside the IDropSource object's QueryContinueDrag implementation. This causes the unresponsive/hang with a callstack like this:

    00000000`0323e908 00007ffd`0ddf54fb USER32!NtUserWaitMessage+0xa
    00000000`0323e910 00007ffd`0ddf8ae2 USER32!DialogBox2+0x212
    00000000`0323e9a0 00007ffd`0ddfad82 USER32!InternalDialogBox+0x132
    00000000`0323ea00 00007ffd`0ddfa51d USER32!SoftModalMessageBox+0xee1
    00000000`0323eb40 00007ffd`0de4805a USER32!MessageBoxWorker+0x2eb
    00000000`0323ecf0 00007ffd`0de480ce USER32!MessageBoxTimeoutW+0xba
    00000000`0323edf0 00000000`08bd8585 USER32!MessageBoxW+0x4e
    00000000`0323ee30 00007ffd`0febd6fa Xxxx::QueryContinueDrag+0x45

    00000000`0323ee60 00007ffd`0febdbb5 ole32!CDragOperation::UpdateTarget+0x182 [d:\blue\com\ole32\ole232\drag\drag.cpp @ 1612]
    00000000`0323ef00 00000000`08bd3799 ole32!DoDragDrop+0x105 [d:\blue\com\ole32\ole232\drag\drag.cpp @ 2619]

    I have found that I can work-around the problem by calling ::ReleaseCapture before presenting modal dialogs, but I was wondering if there is a better way. I would much prefer some solution that is related to the drag/drop implementing component rather than trying to hunt down all the possible places that might present modal dialogs and add ReleaseCapture calls. For example, is there any message that the source window is guaranteed to receive and is there any way to abort the in-progress DoDragDrop call?

    If, in fact, there isn't a better solution, then is there any reason why all modal dialogs should not always call ::ReleaseCapture before they present (and block everyone else)?

    Thanks for any ideas or help,

    Jon Libby

    Wednesday, February 3, 2016 8:49 PM