QueryStatus and Exec both return an HRESULT in ATL. However, the MSDN documentation [1,2] masks the fact. In addition, MSDN does not discuss proper return values.
If Visual Studio were to send a command the function was not able to handle, what is the correct return value: S_OK or S_FALSE. Both are considered success since each is >= to 1. Or should I return E_INVALIDARG, or general failure as in E_FAIL. What I don't want to do is build in adverse behavior so that the add-in breaks in the future when the landscape changes.
I also submitted a suggestion over at connect.microsoft.com to allow developers to submit web pages such as those listed below for review. See http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=425017.
Thanks in advance,
For completeness and the benefit of others who stumble across this thread, here is what I found for QueryStatus Other results are similar. I imagine most on this social do not care about HRESULTs and such, but again, it is for the benefit of others who wander onto the thread. Results were obtained by running Visual Studio under WinDbg.
Visual Studio (msenv.exe) calls AddinPerformQueryStatus to query an add-in status. This was obtained by BP'ing the Add-in's QueryStatus, and then examining the stack:
Next, a BP on the return address (780F8). At the BP, an unassemble:
EAX is the add-in's return value. TEST is an AND that does not store results. JL is jump if less than (signed comparison). TEST EDI, EDI ic checking the return value to see if it is negative. This would translate to 'if( SUCCEEDED(hr) )'. This tells me that S_OK and S_FALSE are not discriminated.
If an error code is returned, a jump is made to 78151:
Above, Visual Studio is trying to find a handler for the command since our add-in did not take responsibility. If a command handler is found, all apears OK. Otherwise, jump to 50078327 and remove the command.
Most interesting from above is 80040104. It is not listed in winerror.h. Searching the web does not get back the phone book as expected. So it appears it may be an undocumented code similar to 'handler not found'. I suspect it is returned from QueryService  or similar API call. What this most likely means is that there is a problem in registration database - that is, the Registry entry for the handler is hosed up.
** Conclusions **
* OK (handled as S_OK)
* fail and remove the command handler from the chain
* attempt to find another handler
- Marked as answer by Wesley Yao Tuesday, March 24, 2009 2:23 AM