none
UIAutomation Managed and Unmanaged API returns contradicting control type: Text or Edit? RRS feed

  • Question

  • Hi,

    I am trying to access UI Automation tree using Managed UI Automation Client API in C++. For certain text controls the call to get_CurrentControlType returns UIA_EditControlTypeId=50004 ("Edit" control) instead of the expected UIA_TextControlTypeId=50020 ("Text" control).

    When accessing the same element using Managed API in C# the AutomationElement.Current.ControlType.ProgrammaticName is "ControlType.Text" and AutomationElement.Current.ControlType.Id is 50020 as expected.

    This only happens for certain controls in certain applications (for example for text in HTML content of IE and Firefox). Static texts in Win32 native applications behave properly, i.e. they are identified as "Text" control type both by Managed and Unmanaged API.

    Is there such a known discrepancy between Managed and Unmanaged UI Automation API?

    Thanks.

    Monday, July 26, 2010 12:10 PM

Answers

  • Hi, Najaryan,

    You are correct in noticing a behavioral difference here.  The COM API's handling of native IAccessibles is different from the managed API.

    The managed API would look at ROLE_SYSTEM_TEXT and convert it to ControlType.Text if the READONLY state bit was set and ControlType.Edit otherwise.

    The COM API just converts it to ControlType.Edit.  If you want to match the older behavior, you can call get_CurrentIsReadOnly and switch the ControlType to ControlType.Text in this case.

    Thanks,
    Mihcael


    This posting is provided "AS IS" with no warranties, and confers no rights.
    Friday, August 6, 2010 4:27 PM

All replies

  • Here are 2 identical programs that should give identical results, however the C# one finds an element of type 50020, C++ finds element type 50004. To give it a try you need to create a trivial HTML file containing text "SPECIAL STRING ON MY HTML PAGE 123ABC", open the file in Internet Explorer then compile and run the program, putting a breakpoint on the blocked commented as "FOUND ELEMENT".

    Any ideas on what might be wrong?

    C# program:

    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Windows.Forms;
    using System.Windows.Automation;
    
    namespace Managed
    {
     static class Program
     {
      /// <summary>
      /// The main entry point for the application.
      /// </summary>
      [STAThread]
      static void Main()
      {
       AutomationElement elem = AutomationElement.RootElement;
       findElem(elem);
    
       //Application.EnableVisualStyles();
       //Application.SetCompatibleTextRenderingDefault(false);
       //Application.Run(new Form1());
      }
    
      static void findElem(AutomationElement elem)
      {
       if (elem.Current.Name == "SPECIAL STRING ON MY HTML PAGE 123ABC")
       {
        // FOUND ELEMENT
        ControlType ct = elem.Current.ControlType; // 50020 as expected
        String lct = elem.Current.LocalizedControlType; // "text" as expected
        return;
       }
    
       foreach (AutomationElement e in elem.FindAll(TreeScope.Children,Condition.TrueCondition))
       {
        findElem(e);
       }
      }
     }
    }
    
    

     

    C++ program:

    // Unmanaged.cpp : Defines the entry point for the console application.
    //
    
    #include "stdafx.h"
    
    #include <string>
    #include <atlbase.h>
    #include <UIAutomation.h>
    #include <UIAutomationClient.h>
    
    IUIAutomation* uia;
    
    void findElem(IUIAutomationElement* elem)
    {
     CComBSTR name;
     if (elem->get_CurrentName(&name)!=S_OK)
     return;
    
     if (name && std::wstring(name)==L"SPECIAL STRING ON MY HTML PAGE 123ABC")
     {
     // FOUND ELEMENT
     CONTROLTYPEID ct;
     elem->get_CurrentControlType(&ct);
     printf("%d",ct); // 50004 instead of expected 50020
     CComBSTR ltype;
     elem->get_CurrentLocalizedControlType(&ltype);
     printf("%s",ltype); // "edit" instead of expected "text"
     return;
     }
    
     CComPtr<IUIAutomationCondition> condition;
     if( uia->CreateTrueCondition(&condition) != S_OK )
     return;
    
     CComPtr<IUIAutomationElementArray> elemArray;
     if( ((IUIAutomationElement*)elem)->FindAll(TreeScope_Children, condition, &elemArray) != S_OK || !elemArray)
     return;
    
     int length=0;
     if( elemArray->get_Length(&length) != S_OK )
     return;
    
     for(int i=0;i<length;i++)
     {
     IUIAutomationElement *element=0;
     if (elemArray->GetElement(i,&element)==S_OK && element)
     {
      findElem(element);
     }
     }
    }
    
    int _tmain(int argc, _TCHAR* argv[])
    {
     CoInitialize(NULL);
    
     HRESULT hr = CoCreateInstance(__uuidof(CUIAutomation), NULL, CLSCTX_INPROC_SERVER, 
      __uuidof(IUIAutomation), (void**)&uia);
     if (FAILED(hr))
     return 0;
    
     IUIAutomationElement* root;
     if (FAILED(uia->GetRootElement(&root)))
     return 0;
    
     findElem(root);
    
    	return 0;
    }
    
    
    
    Tuesday, July 27, 2010 12:56 PM
  • Hi, Najaryan,

    You are correct in noticing a behavioral difference here.  The COM API's handling of native IAccessibles is different from the managed API.

    The managed API would look at ROLE_SYSTEM_TEXT and convert it to ControlType.Text if the READONLY state bit was set and ControlType.Edit otherwise.

    The COM API just converts it to ControlType.Edit.  If you want to match the older behavior, you can call get_CurrentIsReadOnly and switch the ControlType to ControlType.Text in this case.

    Thanks,
    Mihcael


    This posting is provided "AS IS" with no warranties, and confers no rights.
    Friday, August 6, 2010 4:27 PM
  • Michael,

    Thanks! It is indeed as you say.

    Is there a point in attempting to emulate the older behaviour in a newly developed automation tool? This is something that I would want to be sure will not change since the control type will be coded in the end-user scripts. Changes to the UIAutomation API that may change Control Type in the future will break end-user scripts. Is this something we can rely on to be stable or we should not?

    Thanks again,

    Tigran.

    Friday, August 13, 2010 8:52 AM
  • Hi, Tigran,

    It's hard to be perfectly certain about future releases, but the Windows team has no plans to change the Control Type of existing MSAA controls. 

    Thanks,
    Michael


    This posting is provided "AS IS" with no warranties, and confers no rights.
    Friday, August 13, 2010 3:52 PM
  • OK, I understand. Thanks.
    Friday, August 13, 2010 5:11 PM
  • Hi Michael,

    Iam trying to find controls in web page uisng UI Automation. Below link denotes that using

    AriaProperties,AriaRole  properties of UI Automation element we can find the web page controls.

    How ever we are not able to find the AriaProperties,AriaRole  properties for the UI Automation element.

    Please help solve the problem

    http://msdn.microsoft.com/en-us/library/ee684017(v=VS.85).aspx

    Thanks,

    Jamuna

    Monday, February 21, 2011 12:04 PM
  • Hi, Jamuna,

    I would guess that you are using the managed-code API for UI Automation (System.Windows.Automation).  AriaProperties and AriaRole are not supported in the managed-code API at this time.  They are supported in the COM API for UI Automation (IUIAutomation), which can also be used in managed code through COM interop.

    Thanks,
    Michael


    This posting is provided "AS IS" with no warranties, and confers no rights.
    Friday, February 25, 2011 4:50 PM
  • Hi Michael,

    Thanks, I am uisng VS 2008 with VB.Net. please suggest any COM API.

    I am able to navigate all the controls of the web page using walker element (treewalker).

    Issue

    1. I am not able to find the controls inside the frames of the web page.

    2. UI Automation take tool tip value of textbox as Name Property (elementNode.Current.Name) of Automation, most of the time tool tip property is empty.

    For that I need to get the ID of the web page control which is unique to identify the control.

    Please help to solve the problem

    Regards,

    Jamuna

     

     

    Wednesday, March 9, 2011 1:29 PM
  • Hi Michael,

    One more question how to check whether the Invoke action of Invoke Pattern has completed successfully.

    Similarly for every action of each control pattern, I need to check for success or failure or completed status.

     

    Thanks,

    Jamuna

    Friday, March 11, 2011 12:01 PM
  • Hi, Jamuna,

    The COM API starts with the IUIAutomation interface, and is documented here:
    http://msdn.microsoft.com/en-us/library/ee684021(VS.85).aspx

    I have a sample that uses the COM API from managed code here; take a look at the unit tests to see how this is done.  Follow the link in the first paragraph to the samples.
    http://blogs.msdn.com/b/winuiautomation/archive/2010/12/08/uia-custom-patterns-part-1.aspx

    Almost every control pattern has a corresponding event: Invoke pattern should cause the UIA_InvokedEventId event.  The ExpandCollapse pattern should cause a PropertyChange event for the UIA_ExpandCollapseExpandCollapseStatePropertyId.  Check the documentation for each pattern for more details.

    Thanks,
    Michael


    This posting is provided "AS IS" with no warranties, and confers no rights.
    Friday, March 11, 2011 4:49 PM
  • Hi Michael,

    Thank you

    http://uiacomwrapper.codeplex.com/

    In the above link, you have provided the UI Comwrapper class.

    Can I use this to get the Aria Properties and Aria roles.

    Using that Can I get the ID of Web Controls like I need to get the id value "txt1" of Automation Element<input id="txt1" type="text"></input>.

    If possible using above com wrapper class, please help for some sample code for fetching the id properties.

    Regards,

    Jamuna

    Monday, March 14, 2011 2:28 PM
  • Hi, Jamuna,

    You can certainly use the UIA Com Wrapper class, but there isn't support in Internet Explorer to expose the ID of web controls through UIA.  (This is a limitation in Internal Explorer.) 

    Thanks,
    Michael


    This posting is provided "AS IS" with no warranties, and confers no rights.
    Monday, March 14, 2011 3:23 PM
  • Hi Michael,

    Thanks

    However I am able to view CurrentAriaRole properties in the interface.

    Will that support for my requirement

    Dim

     

    elemeent1 As IUIAutomationElement

    elemeent1.CurrentAriaRole

    Regards,

    Jamuna

     

     

    Tuesday, March 15, 2011 6:15 AM
  • Hi all,

     

    Could you please open a new topic for new issues.

     

    Thanks.

    Tuesday, March 15, 2011 8:36 AM