The following code was added to a Win32 application's WndProc to hit test the minimize, maximize, and close buttons. The buttons are detected if Desktop Composition is disabled, but not when it is enabled. Other non-client regions are detected when Desktop Composition is enabled. Does anyone know of a work-around for this issue?
lResult = DefWindowProc(hWnd, message, wParam, lParam);
OutputDebugString(L"HTCLOSE\n"); //detected if Desktop composition is disabled
else if (lResult==HTMINBUTTON)
OutputDebugString(L"HTMINBUTTON\n"); //detected if Desktop composition is disabled
else if (lResult==HTMAXBUTTON)
OutputDebugString(L"HTMAXBUTTON\n"); //detected if Desktop composition is disabled
else OutputDebugString(L"HT other\n");
Instead of using DefWindowProc, when composition is enabled use DwmDefWindowProc, which will perform NC hit testing for you for the buttons only ( but works even if you've set your nc area to 0 and used DwmExtendFrameIntoClientArea to get min/max/close buttons on your client area). If it returns without detecting a btn, pass it along to DefWindowProc as well for caption, sizing, etc testing.
Thanks for the suggestion, but I still can't detect the buttons. I tried the following - the documentation for DwmDefWindowProc is a bit thin so I'm not sure if this is what you meant:
hr = DwmIsCompositionEnabled( &bDwmCompositionEnabled );
if ( SUCCEEDED( hr ) && bDwmCompositionEnabled )
hr = DwmDefWindowProc( hWnd, message, wParam, lParam, &lResult );
if ( SUCCEEDED( hr ) )
if ( lResult==HTCLOSE )
OutputDebugString( L"DWM HTCLOSE\n" ); //never detected
OutputDebugString( L"DWM Not HTCLOSE\n" );
When desktop composition is enabled there are no WM_NCHITTEST messages when the arrow is over the buttons so the DwmDefWindowProc never gets called. There are messages over other non-client areas - DwmDefWindowProc returns S_OK but the debugger shows lResult with the uninitialized value of 0xcccccccc. (note: I am using RC1 build 5600)
Actually, I test for HTCLIENT rather than the return of DwmDefWindowProc, but that's pretty much exactly what I meant.
So you have the standard titlebar - you don't have the frame extended in or drawn yourself? I haven't used DwmDefWindowProc under those circumstances.
Right now, my usage is (when Dwm enabled)
1) respond to wm_nccalcsize and set nc top area to 0 (client starts at the top)
2) call DwmExtendFrameIntoClientArea, extending it down the height of a titlebar.
3) respond to wm_nchittest and pass it to DwmDefWindowProc first. if htclient is passed back I do my own testing to see if it should be htcaption
Note that step 2 above *will* show the control menu (icon), caption text, and min/max/close buttons as if it were in the standard NC area. if you pass wm_nchittest to dwmdefwindowproc it will handle the "glow" and hit testing of the buttons for you.
However, I'm not sure how to go about testing in your scenario - I'm afraid I'm stumped! sorry to lead you down the wrong path. I'm still struggling with proper handling of tricks in the NC area under vista as well (drawing, etc)
Thanks for the info on the usage of DwmDefWindowProc - that's the first description of its usage that I've seen.
I am getting the impression that when the buttons are in the non-client area, DWM is gobbling up the WM_NCHITTEST messages to make the buttons glow and not passing them along to the WndProc. If this is true it sounds like a design flaw to me - Vista should either pass them along or use a different message so that an app can still hit test them when desktop composition is enabled.