none
GetRuntimeId() returns empty arrays RRS feed

  • Question

  • If i understand it correct GetRuntimeId should never return zero-sized SAFEARRAY since MSDN says that GetRuntimeId "Retrieves the unique identifier assigned to the UI element.".

    However when I execute the following C++ code I can see hundreds of elements that return empty SAFEARRAY as their RuntimeId. Executing equivalent C# managed code does not return empty RuntimeId arrays so this looks like a bug in native UI Automation Client API.

    Is there a workaround to this problem and if not where should I report this?

    #include "stdafx.h"
    
    #include <time.h>
    #include <conio.h>
    
    #include <string>
    #include <atlbase.h>
    #include <UIAutomation.h>
    #include <UIAutomationClient.h>
    
    CComPtr<IUIAutomation> uia;
    CComPtr<IUIAutomationTreeWalker> walker;
    int childCount;
    
    void findElem(IUIAutomationElement* elem,int level)
    {
     SAFEARRAY *rid;
     if (SUCCEEDED(elem->GetRuntimeId(&rid)))
     {
      LONG ubound;
      if (SUCCEEDED(SafeArrayGetUBound(rid,1,&ubound)) && ubound<=0)
      {
       CComBSTR name,className;
       int pid;
       if (SUCCEEDED(elem->get_CurrentName(&name)) && 
         SUCCEEDED(elem->get_CurrentClassName(&className)) &&
         SUCCEEDED(elem->get_CurrentProcessId(&pid)))
       {
        wprintf(L"zero-sized Runtime ID array, name=%s, className=%s, PID=%d\n",name,className,pid);
       }
      }
     }
    
     CComPtr<IUIAutomationElement> child;
     if (walker->GetFirstChildElement(elem,&child)!=S_OK || !child)
      return;
    
     do
     {
      static clock_t c;
      clock_t c1 = clock();
      clock_t d = c1-c;
      if (d>1000)
       printf("%d\n",1000*d/CLOCKS_PER_SEC);
      c=c1;
    
      childCount++;
      findElem(child,level+1);
      CComPtr<IUIAutomationElement> sibling;
      if (walker->GetNextSiblingElement(child,&sibling)!=S_OK)
       break;
      child = sibling;
     } while (child);
    }
    
    DWORD WINAPI testUIA(LPVOID param)
    {
     clock_t c = clock();
    
     childCount = 0;
    
     HRESULT hr;
     if (param==0)
      hr = CoInitialize(NULL);
     else
      hr = CoInitializeEx(NULL,COINIT_MULTITHREADED);
    
     uia=0;
     hr = CoCreateInstance(__uuidof(CUIAutomation), NULL, CLSCTX_INPROC_SERVER, 
       __uuidof(IUIAutomation), (void**)&uia);
     if (FAILED(hr))
      return 0;
    
     walker = 0;
     if (FAILED(uia->get_ControlViewWalker(&walker)))
      return 0;
    
     IUIAutomationElement* root;
     if (FAILED(uia->GetRootElement(&root)))
      return 0;
    
     findElem(root,0);
    
     clock_t d = clock()-c;
     printf("total %f sec, count=%d, avg %d us\n",(float)(d)/CLOCKS_PER_SEC,childCount,
      int((float)(d)/CLOCKS_PER_SEC/childCount*1e6));
    
     CoUninitialize();
    
     return 0;
    }
    
    int _tmain(int argc, _TCHAR* argv[])
    {
     printf("Testing in main thread STA...\n");
     testUIA(0);
    
     printf("Press any key to exit.\n");
     _getch();
    
    	return 0;
    }
    
    
    

     

    Saturday, October 16, 2010 12:02 PM

Answers

  • Hi, Najaryan,

    The nodes without runtime IDs are almost certainly MSAA nodes that are being bridged into UIA.  There isn't an algorithm to compute a unique ID for an MSAA node that is not cost-prohibitive (like walking to the root of the desktop).  Managed-code UIA does the expensive algorithm, and we received a lot of complaints about its performance.  We did not repeat that design in the native-code client.

    You can confirm this hypothesis by looking at the ProviderDescription property and seeing if it includes MSAA Proxy.

    The workaround depends on your goal.  Runtime IDs are generally used to tell if two elements are identical.  If that's your goal, then you'll need to handle the case where both elements have empty runtime IDs.  At that point, if you choose to do so, you can do the expensive algorithm yourself, walking up to the root (or until you find a parent with a runtime ID) and then comparing the parent chains.  This is hard work, which is one reason that we encourage new providers to use UIA.

    Thanks,
    Michael


    This posting is provided "AS IS" with no warranties, and confers no rights.
    Friday, October 29, 2010 4:43 PM

All replies

  • Hi, Najaryan,

    The nodes without runtime IDs are almost certainly MSAA nodes that are being bridged into UIA.  There isn't an algorithm to compute a unique ID for an MSAA node that is not cost-prohibitive (like walking to the root of the desktop).  Managed-code UIA does the expensive algorithm, and we received a lot of complaints about its performance.  We did not repeat that design in the native-code client.

    You can confirm this hypothesis by looking at the ProviderDescription property and seeing if it includes MSAA Proxy.

    The workaround depends on your goal.  Runtime IDs are generally used to tell if two elements are identical.  If that's your goal, then you'll need to handle the case where both elements have empty runtime IDs.  At that point, if you choose to do so, you can do the expensive algorithm yourself, walking up to the root (or until you find a parent with a runtime ID) and then comparing the parent chains.  This is hard work, which is one reason that we encourage new providers to use UIA.

    Thanks,
    Michael


    This posting is provided "AS IS" with no warranties, and confers no rights.
    Friday, October 29, 2010 4:43 PM
  • Thanks for replying.

    Yes, the ProviderDescription says MSAA Proxy.

    I am using Runtime IDs to uniquely identify elements across process boundaries. I am using a helper process to handle Automation events as I mentioned in this post . The helper process notifies the main process about Automation structure changes and uses Runtime ID to pinpoint changed elements. 

    I am not sure how to handle empty Runtime ID cases myself. I suppose you suggest to traverse up the tree and compose my own ID based on some parent-child relationship information. The easiest thing that comes to mind is using child index within its parent so the IDs would be arrays of integers specifying positions of children. For example ID [4,2] would identify 2nd child of 4th child of root.

    The problem with this approach however is that any change of children ordering would render the IDs useless. I fail to see how it is possible to construct unique IDs that remain unchanged during a run-time using information available to UI Automation Client. I believe this is only possible to do in Provider side of things where the provider knows what properties of an element are sufficient to uniquely identify it among its siblings and are unchanged during the lifetime of the element (HWND is a good example that is used by UI Automation).

    Please correct me if I am wrong.
    • Edited by najaryan Saturday, October 30, 2010 6:28 AM fixed formatting
    Saturday, October 30, 2010 6:28 AM
  • I would separate out two questions here: (1) How can one identify an element without a runtime ID in a stable system, and (2) How does that method stand up to changes?

    I think your answer to (1) is correct - using child indices seems reasonable. 

    You are also correct that (2) is extremely hard.  This is one of the reasons we introduced UIA -- MSAA has a series of limitations, including this one, that cannot be overcome.  You might try a combination of MSAA properties: if the accNames are the same, and the accRoles are the same, and the accStates are the same, and the accLocations are the same, then it is probably the same element.  But even this is only a heuristic.  As you said, the right answer is for the provider to do it, which is what UIA does. 

    But when UIA is representing an MSAA node, UIA doesn't have any more data than you do -- it cannot create something out of nothing. 

    Thanks,
    Michael


    This posting is provided "AS IS" with no warranties, and confers no rights.
    Monday, November 1, 2010 6:01 PM
  • Michael, thanks. I understand.

    I actually have managed to avoid the problem that was caused by empty runtime IDs in my particular use case.

    Thanks again for your follow up, I appreciate it. 

    Monday, November 1, 2010 7:05 PM