none
Problem with ListItem Checkbox using Automation UI

    Question

  • Hi

    I am automating a simple third party application which displays a ListItem Checkbox.
    I am unable to obtain the TogglePattern for this control and UISPy indicates that no such pattern exists for this control.
    Although I can select, focus and check the item via the keyboard or even a mouse click, I am unable to ascertain if the check actually occurred as
    I am unable to retrieve the Toggle Status for this control.

    UISpy reports that the control is of type ControlType.ListItem with a localised type of list item. It is contained with a List of type Class TNewCheckListBox, ControlType.List and localised type of list. The internal settings are:-

    CanSelectMultiple = false
    IsSelectionRequired = true

    Using Microsofts Automation UI, is there a way I can retrieve the Toggle state as so far I am drawing a blank with this.

    Thanks

     

    Monday, October 24, 2011 10:47 AM

Answers

  • Hi Ant,

    To give you some background, UI Automation (UIA) was introduced around the time of Windows Vista as part of the .NET Framework. That version of UIA was a managed API, and continues to be available with the .NET Framework. In Windows 7, an unmanaged version of the UIA API was introduced as part of Windows, separate from the managed .NET version. That Windows API had some enhancements which were not available in the .NET version, and that means that in general, if the results between the two APIs are not the same, the unmanaged API provides more useful information. (This technically isn't always the case, but typically is, and is in the case of the Toggle Pattern for you.)

    The newest versions of the Windows SDK tools use the unmanaged version of UIA. The two tools that I use all the time are Inspect, (to examine the accessible properties and patterns of UI), and AccEvent, (to see what events are being raised by UI.) I use both of these tools in UIA mode. (They both have legacy MSAA modes which are of no interest to UIA clients.) UISpy is an older tool built on the the managed .NET API, and so I don't use that myself.

    I would strongly recommend that if it's practical, you switch to using the unmanaged UIA API introduced with Windows 7. If your client app is written in C#, then you have a couple of options for using the unmanaged API.

    (i) Use the tlbimp.exe tool to generate a COM wrapper around the unmanaged UIA API, such that you can call the API from C#. A while ago I built a C++ sample showing some of the caching power of UIA, up at http://code.msdn.microsoft.com/Windows-7-UI-Automation-9131f729. I then rebuilt exactly the same app in C#, http://code.msdn.microsoft.com/Windows-7-UI-Automation-0625f55e. That C# app uses the interface generated by tlbimp.

    I also built a couple more C# samples, http://code.msdn.microsoft.com/Windows-7-UI-Automation-6390614a and http://code.msdn.microsoft.com/Windows-7-UI-Automation-433a961f. I was particularly pleased with the latter, as that was a real-world app I built following a request from someone asking for a way to have the prediction keys on the on-screen keyboard spoken, in order for it to be more useful to her students. It was natural to use UIA inside this app.

    (ii) Alternatively, if you already have a fair bit of C# code built against the .NET UIA API, you probably don't want to rewrite that to call the COM wrapper generated in tlbimp.exe. If so, you can use another wrapper which specifically wraps the Windows 7 UIA API such that it looks like the managed UIA API. The wrapper is available at http://uiacomwrapper.codeplex.com/. Hopefully by using that wrapper your existing C# code can call into the Windows unmanaged UIA API with little or no changes to your code.

    If you want more details on either of the above options, let me know. But by doing one of these, you should find your code sees the Toggle Pattern on the listitems. By the way, if I ever find my client app doesn't see something I expect, then I check things out with Inspect. If Inspect sees something that my client app doesn't, then I usually need to change my client app. But if Inspect doesn't see what I expect either, then this usually means what I expected isn't being exposed by whatever app I'm interacting with. There's nothing UIA can do to make an inaccessible app accessible. UIA is really a means for channeling data from a provider app to a client app, and so if the provider doesn't expose some pattern, property or event, then UIA can't propagate it through to your client. In those cases, I typically use Inspect and AccEvent to see what patterns, properties and events are exposed by the provider, and try to figure out some way for my client to achieve what it needs to with those.

    Regarding the SelectionItemPattern question, I'm not really sure where you're using this. Is it to determine whether the listitem's checkbox is checked? If so, perhaps you could check the ToggleState property of the listitem instead. I would expect that property to change as the checkbox is toggled.

    Thanks,

    Guy

    • Marked as answer by CygnusAlpha Tuesday, November 01, 2011 3:35 PM
    Wednesday, October 26, 2011 2:23 PM

All replies

  • Hi,

    If you use the latest Inspect SDK tool to examine the ListItem UI, does the toggle pattern show up? UISpy uses the .Net managed version of the UI Automation API, but Inspect uses the unmanaged version introduced in Windows 7. Sometimes the results shown by UISpy and Inspect can be different, so I'd be interested to know if Inspect reports the data you need.

    Thanks,

    Guy

    Tuesday, October 25, 2011 3:53 PM
  • Hi Guy

    You are quite correct. Using Inspect does indeed show the Toggle Pattern and Toggle State for this control.
    However, navigating through the elements from the List itself to the single List Item and I cannot get the Toggle Pattern from this control.

    If I can see this pattern listed via Inspect then I assume that this pattern is available and therefore I should be able to retrieve it via
    code.

    Most odd indeed so any further help appreciated please.

    Regards

    Ant

    Tuesday, October 25, 2011 10:00 PM
  • Guy

    Further to this the only pattern I get is

    SelectionItemPatternIdentifiers

    and this only contains IsSelected and SelectionContainer.

    I'm using Managed C# code to do this, I'll try some Native code and see if I get any differences.

    Thanks

    Ant

    Wednesday, October 26, 2011 10:45 AM
  • Hi Ant,

    To give you some background, UI Automation (UIA) was introduced around the time of Windows Vista as part of the .NET Framework. That version of UIA was a managed API, and continues to be available with the .NET Framework. In Windows 7, an unmanaged version of the UIA API was introduced as part of Windows, separate from the managed .NET version. That Windows API had some enhancements which were not available in the .NET version, and that means that in general, if the results between the two APIs are not the same, the unmanaged API provides more useful information. (This technically isn't always the case, but typically is, and is in the case of the Toggle Pattern for you.)

    The newest versions of the Windows SDK tools use the unmanaged version of UIA. The two tools that I use all the time are Inspect, (to examine the accessible properties and patterns of UI), and AccEvent, (to see what events are being raised by UI.) I use both of these tools in UIA mode. (They both have legacy MSAA modes which are of no interest to UIA clients.) UISpy is an older tool built on the the managed .NET API, and so I don't use that myself.

    I would strongly recommend that if it's practical, you switch to using the unmanaged UIA API introduced with Windows 7. If your client app is written in C#, then you have a couple of options for using the unmanaged API.

    (i) Use the tlbimp.exe tool to generate a COM wrapper around the unmanaged UIA API, such that you can call the API from C#. A while ago I built a C++ sample showing some of the caching power of UIA, up at http://code.msdn.microsoft.com/Windows-7-UI-Automation-9131f729. I then rebuilt exactly the same app in C#, http://code.msdn.microsoft.com/Windows-7-UI-Automation-0625f55e. That C# app uses the interface generated by tlbimp.

    I also built a couple more C# samples, http://code.msdn.microsoft.com/Windows-7-UI-Automation-6390614a and http://code.msdn.microsoft.com/Windows-7-UI-Automation-433a961f. I was particularly pleased with the latter, as that was a real-world app I built following a request from someone asking for a way to have the prediction keys on the on-screen keyboard spoken, in order for it to be more useful to her students. It was natural to use UIA inside this app.

    (ii) Alternatively, if you already have a fair bit of C# code built against the .NET UIA API, you probably don't want to rewrite that to call the COM wrapper generated in tlbimp.exe. If so, you can use another wrapper which specifically wraps the Windows 7 UIA API such that it looks like the managed UIA API. The wrapper is available at http://uiacomwrapper.codeplex.com/. Hopefully by using that wrapper your existing C# code can call into the Windows unmanaged UIA API with little or no changes to your code.

    If you want more details on either of the above options, let me know. But by doing one of these, you should find your code sees the Toggle Pattern on the listitems. By the way, if I ever find my client app doesn't see something I expect, then I check things out with Inspect. If Inspect sees something that my client app doesn't, then I usually need to change my client app. But if Inspect doesn't see what I expect either, then this usually means what I expected isn't being exposed by whatever app I'm interacting with. There's nothing UIA can do to make an inaccessible app accessible. UIA is really a means for channeling data from a provider app to a client app, and so if the provider doesn't expose some pattern, property or event, then UIA can't propagate it through to your client. In those cases, I typically use Inspect and AccEvent to see what patterns, properties and events are exposed by the provider, and try to figure out some way for my client to achieve what it needs to with those.

    Regarding the SelectionItemPattern question, I'm not really sure where you're using this. Is it to determine whether the listitem's checkbox is checked? If so, perhaps you could check the ToggleState property of the listitem instead. I would expect that property to change as the checkbox is toggled.

    Thanks,

    Guy

    • Marked as answer by CygnusAlpha Tuesday, November 01, 2011 3:35 PM
    Wednesday, October 26, 2011 2:23 PM
  • Hi Guy

    Thank you for your in-depth explanation. I was not aware of the Managed Automation UI's limitations.
    From what you have explained, it does sound feasible that what I need to do is to use the Native version for such controls.
    Up until now, the managed library has been very successful. I will look into your examples and tools in order to resolve this issue.

    Thanks

    Ant

    Tuesday, November 01, 2011 3:35 PM
  • For anyone who reads this

    I've been programming using the UIAComWrapper for few weeks. I reached now a point where I want to listen to the UIA events.
    I found out that the UIAComWrapper did not raise any InvokedPattern.InvokedEvent on a button it managed to locate in the UIA tree.
    When I switched back to using the original C#  UIAutomationClient reference, it did raise all the proper events I subscribed to. The problem is that the latter does not reveal the full UIA tree as I asked here and got reply form Guy Barker - http://social.msdn.microsoft.com/Forums/en-US/windowsaccessibilityandautomation/thread/64259345-d416-41a7-8dba-7557c30bb606/

    So Im in a funny situation now:
    - the C# UIAutomationClient does not show the whole UIA tree but is more reliable as far as UIA events
    - the UIAComWrapper  shows much more of the tree elements, but not the events

    Im trying to avoid rewriting my project in C++. 
    So the next thing Im going to try is using the  tlbimp.

    Any comments about my decision, am I going the right way would be mostly appreciated

    THANKS!

     


    lior

    Thursday, September 20, 2012 3:40 PM