none
InvokePattern with hyperlinks

    Question

  • I apologize if this is obvious, I did research this before I asked, and didn't find the answer. The UI Automation documentation indicates that hyperlinks support the Invoke event. I am using IE9, Win 7 64, and Accessible Event Watcher v 7.1.  I see invoke events with thick client apps (e.g. Calculator Buttons) but not hyperlinks.  Could someone point me to a resource where I can get answers to the following questions? : Do I misunderstand and this is not supported? Do I have to do something to enable them? Is there an alternative way to track hyperlink clicks? Is this a security issue related to access levels?

    mardi 6 mars 2012 04:55

Réponses

  • Hi,

    Your understanding here is correct. You’re right to expect that Invoke events should be raised when the hyperlink is clicked, and the fact that you're not receiving them is not due to anything you’re doing. Having said that, I'm not going to be able to say anything to help you here.

    I just examined a number of buttons and hyperlinks in various UI using the Inspect SDK tool. The control types for the controls I looked at were reported to be UIA_ButtonControlTypeId and UIA_HyperlinkControlTypeId, and Inspect also showed that the controls claim to support the Invoke pattern, (as shown in the image below.) As as test, while I had the control highlighted by Inspect, I went to Inspect's "Action" and could see Invoke.Invoke listed. (The Action menu lists all the methods that can be called on the patterns supported by the control.) So I clicked Invoke.Invoke and the link reacted as appropriate, which means the Invoke pattern is at least partially working. This is good news, as the ability to programmatically invoke a link is essential in many assistive technology scenarios.

    I then used the AccEvent SDK tool to report any UIA_Invoke_InvokedEventId events raised (by any element anywhere) when I physically clicked on the link. When I did this, I did see an Invoke event being raised. (The image below shows the related AccEvent settings and output.) The interesting thing here is that AccEvent can show no details about the element that raised the event at all. When AccEvent gets the event, it tries to go back to the source element and get specific data from it. But in this case, the source element no longer exists, as the related UI has been destroyed and new UI created as a result of clicking the link. (This might be avoidable by UIA clients who using caching to request that data is gathered at the time the event is raised, rather than trying to go back to the source element later.)

    I then repeated this test in IE9. On elements that claim to support the Invoke pattern, I could programmatically invoke them as expected. But when I physically clicked on them, no Invoke events were raised. I don't know whether this is due to IE or UIA itself, but I don't think there's anything you'll be able to do on the client side to work around this.

    I think in practice this probably isn't a major concern in many scenarios because for assistive technology users, it's all the things that happen as a result of invoking the element that are most important, rather than the details of the invoke itself. For example, a screen reader user needs to get details of the links shown and needs to be able to invoke them, but once it's invoked the user's interested in the details of the destination page that's being presented.

    Sorry I can't be more help here.

    Guy


    mardi 6 mars 2012 14:33

Toutes les réponses

  • Hi,

    Your understanding here is correct. You’re right to expect that Invoke events should be raised when the hyperlink is clicked, and the fact that you're not receiving them is not due to anything you’re doing. Having said that, I'm not going to be able to say anything to help you here.

    I just examined a number of buttons and hyperlinks in various UI using the Inspect SDK tool. The control types for the controls I looked at were reported to be UIA_ButtonControlTypeId and UIA_HyperlinkControlTypeId, and Inspect also showed that the controls claim to support the Invoke pattern, (as shown in the image below.) As as test, while I had the control highlighted by Inspect, I went to Inspect's "Action" and could see Invoke.Invoke listed. (The Action menu lists all the methods that can be called on the patterns supported by the control.) So I clicked Invoke.Invoke and the link reacted as appropriate, which means the Invoke pattern is at least partially working. This is good news, as the ability to programmatically invoke a link is essential in many assistive technology scenarios.

    I then used the AccEvent SDK tool to report any UIA_Invoke_InvokedEventId events raised (by any element anywhere) when I physically clicked on the link. When I did this, I did see an Invoke event being raised. (The image below shows the related AccEvent settings and output.) The interesting thing here is that AccEvent can show no details about the element that raised the event at all. When AccEvent gets the event, it tries to go back to the source element and get specific data from it. But in this case, the source element no longer exists, as the related UI has been destroyed and new UI created as a result of clicking the link. (This might be avoidable by UIA clients who using caching to request that data is gathered at the time the event is raised, rather than trying to go back to the source element later.)

    I then repeated this test in IE9. On elements that claim to support the Invoke pattern, I could programmatically invoke them as expected. But when I physically clicked on them, no Invoke events were raised. I don't know whether this is due to IE or UIA itself, but I don't think there's anything you'll be able to do on the client side to work around this.

    I think in practice this probably isn't a major concern in many scenarios because for assistive technology users, it's all the things that happen as a result of invoking the element that are most important, rather than the details of the invoke itself. For example, a screen reader user needs to get details of the links shown and needs to be able to invoke them, but once it's invoked the user's interested in the details of the destination page that's being presented.

    Sorry I can't be more help here.

    Guy


    mardi 6 mars 2012 14:33
  • An additional note for people using AccEvent...

    I notice that in the AccEvent image I included earlier, it shows "hr = 0x80040201 - An event was unable to invoke any of the subscribers". The error 0x80040201 in this case actually means that the UIA element is not available. The error's defined in UIAutomationCoreApi.h...

    #define UIA_E_ELEMENTNOTAVAILABLE    0x80040201

    Unforuntately this error has the same value as an unrelated error defined in winerror.h...

    #define EVENT_E_ALL_SUBSCRIBERS_FAILED   _HRESULT_TYPEDEF_(0x80040201L)

    This leads to confusion, because UIA clients will periodically encounter the error value 0x80040201.

    Guy

    mardi 6 mars 2012 14:41
  • Wow. Many thanks for the thorough response. I do appreciate the effort you went through for this. As an aside the WinEvents mode of AccEvent doesn't get an invoke event either. I was hopeful since that seemed to be a lower level interface. Unfortunately, for my use case I really need to see the invoke event. 

    What version of IE were you able to see the invoke (with the error)?

    mercredi 7 mars 2012 01:57
  • Hi,

    Unfortunately I've not been able to get the Invoke event raised from IE at all, (I was using IE9). This Invoke event I saw in AccEvent was raised by the Control Panel when I clicked a link in there. By doing that I knew I had AccEvent set up right to report Invoke events, and so the fact that I didn't see them when clicking the links in IE9 meant that they either weren't being raised by IE, or UIA wasn't propagating them through to the client. That was an interesting test that you ran, to see if the EVENT_OBJECT_INVOKED winevent was being raised. Given that that winevent isn't being raised, I think this is an IE9 issue rather than UIA.

    I tried looking for any other events raised when the link's clicked, in case it's practical to consider that some other series of events implies that a link's probably been clicked, but I didn't see anything like this at all. It really looks like there's nothing available to help recognize the link click.

    I can't think of anything helpful to suggest here if you really need to know the link's been clicked. Depending on your needs, perhaps there are some inferences that can be made, but that's far from ideal. For example, say you already track the element beneath the mouse cursor and the element with keyboard focus. (The Inspect tool does this.) This means as the mouse and keyboard focus moves, you already know some details about the related element, (perhaps say its name and bounding rectangle). When you get that data, you could also get the IsInvokePatternAvailable property. If a link is clicked, then a few StructureChange events are raised beneath the element for the main IE window. In my tests, it looks like whenever I click a link and the page changes, I get a StructureChangeType_ChildAdded event for the element whose class is "Internet Explorer_Server". So perhaps you could assume that if you do get such an event when the previous element to be reached via the mouse or keyboard had the Invoke pattern, then there's a reasonable chance that the invoking of the element led to the Internet Explorer_Server element's tree structure changing.

    It seems pretty unlikely that something like the above approach is really any use in practice, but I can't think of a good solution for a client app that really wants to know about the click versus only needing to handle all that happens in response to the click.

    Guy

    mercredi 7 mars 2012 03:10