none
Problem finding AutomationElement in PowerShell script

    Question

  • Hi all,

     

    I'm trying to manipulate the UI of notepad.exe, using the UI Automation framework from a powershell script. I've been meaning to try this for a while, since it seems like it could be a great way to write and distribute lightweight test scripts.

     

    But I've run into a roadblock - it seems that very similar UI Automation code behaves differently depending on whether I write it in C# or powershell. I'm hoping someone can shed some light on this.

     

    I've tried this on XP SP2 and Vista Ultimate and the behavior is the same on both.

     

    Here's what my script is trying to do:

     

    1) Start notepad and assign its Process object to a variable

    2) Get the notepad window's AutomationElement using the hwnd of the Process

    3) Create a PropertyCondition for the men bar's AutomationId, which is "MenuBar"

    4) Use the condition to call FindFirst() on the main window's AutomationElement and get an element for the menu bar

     

    In C#, this is easy; retrieving the menu bar element works fine. But in Powershell I have not been able to retrieve the menu bar element - I can get the main notepad window element, but calling FindFirst() on it always fails to find the menu bar.

     

    The following C# code (mostly taken from the March '07 MSDN mag article on UI Automation) was my starting point - I've just attempted to translate it to powershell. The C# code always works; when it runs, 'menuBarElement' always contains the correct element, and it can be used to obtain the child elements of the menu bar.

     

    Code Snippet

    // Start notepad and give it a moment to initialize

    Process proc = Process.Start("notepad");

    Thread.Sleep(1000);

    // Attach to the main window.

    AutomationElement rootElement = AutomationElement.FromHandle(proc.MainWindowHandle);

    // Create a condition to find the menu bar

    PropertyCondition menuBarFind = new PropertyCondition(AutomationElement.AutomationIdProperty, "MenuBar");

    // Find the menuBar, which is a child of the main window.

    AutomationElement menuBarElement = rootElement.FindFirst(TreeScope.Children, menuBarFind);

    // Create condition to find the find the File menu element

    PropertyCondition fileMenuFind = new PropertyCondition(AutomationElement.AutomationIdProperty, "Item 1");

    // Find the element and print its AutomationId property

    AutomationElement fileMenuElement = menuBarElement.FindFirst(TreeScope.Children, fileMenuFind);

    System.Console.WriteLine(fileMenuElement.Current.AutomationId);

    Thread.Sleep(10000);

     

    And here is the powershell version. Everything works the same, except that menuBar is always null at the end, so calling FindFirst() on it at the last line fails:

     

    Code Snippet

    # PowerShell example

    #

    # Load UI Automation assemblies

    [Reflection.Assembly]::LoadFile('C:\Program Files\Reference Assemblies\Microsoft\Framework\v3.0\UIAutomationClient.dll')

    [Reflection.Assembly]::LoadFile('C:\Program Files\Reference Assemblies\Microsoft\Framework\v3.0\UIAutomationTypes.dll')

    # Start notepad and get process object

    $proc = [diagnostics.process]::start("notepad")

    # Use window handle to attach to the main window

    $rootElement = [Windows.Automation.AutomationElement]::FromHandle($proc.MainWindowHandle)

    # Create condition to find the menu bar

    $menuBarFind = new-object Windows.Automation.PropertyCondition -argumentlist ([Windows.Automation.AutomationElement]::AutomationIdProperty), "MenuBar"

    # Find the menu bar within the main window

    $menuBarElement = $rootElement.FindFirst(([System.Windows.Automation.TreeScope]::Children), $menuBarFind)

    # Create condition to find file menu

    $fileMenuFind = new-object -typename Windows.Automation.PropertyCondition -argumentlist ([Windows.Automation.AutomationElement]::AutomationIdProperty), "Item 1"

    # Find the File menu - this is where the script fails, because $menuBarElement is null

    $fileMenuElement = $menuBarElement.FindFirst(([System.Windows.Automation.TreeScope]::Children, $fileMenuFind))

     

    So does anyone know what might be going on? I suspected that perhaps the elements are being filtered, or the scope/view of the window available to powershell may be different. But I don't know.

    Thursday, June 07, 2007 11:43 PM

All replies

  • Did you ever find the answer to this- I'm trying to do the same thing.
    Wednesday, July 25, 2007 1:51 AM
  • I do not know if this helps. But what is the ApartmentState of a your thread in the powershell? If it is MTA maybe try it from a new thread where you set the ApartmentState to STA.
    Wednesday, July 25, 2007 7:44 AM
  • There are two problems here:

     

    First – the big problem, as mentioned by the previous poster. The UI Automation APIs only work from an STA thread and PowerShell always runs scripts MTA. If you run the C# example code compiled MTA, you’ll see the same problem you see with the PowerShell script.  There are workarounds for this  but they’re pretty clunky. This is something we’re going to fix in V2. In the mean time, see Krishna’s blog entry:

    http://blogs.msdn.com/powershell/archive/2007/03/23/thread-apartmentstate-and-powershell-execution-thread.aspx

     

    The second issue appears to be a problem with how PowerShell handles value types. In order to make the script work, I had to create a helper class:

     

    Code Snippet

        public static class AutomationHelper

        {

            public static object GetRoot(System.IntPtr mwh)

            {

                return System.Windows.Automation.AutomationElement.FromHandle(mwh);

            }

        }

     

     

     

    to get the root automation element. However, once I had the root element, everything worked fine after that. I used Lee Holmes’s csharp inline-compiler to build the helper class. A working script (assuming you can run it STA) looks like:

     

    Code Snippet

    ####################################################################

    #

    # PowerShell UIAutomation example

    #

     

    #

    # Load UI Automation assemblies

    #

    $AssemblyPath = "$env:programfiles\Reference Assemblies\Microsoft\Framework\v3.0"

    [void] [Reflection.Assembly]::LoadFile("$AssemblyPath\UIAutomationClient.dll")

    [void] [Reflection.Assembly]::LoadFile("$AssemblyPath\UIAutomationTypes.dll")

     

    #

    # Define a helper function...

    #

    csharp @'

        public static class AutomationHelper

        {

            public static object GetRoot(System.IntPtr mwh)

            {

                return System.Windows.Automation.AutomationElement.FromHandle(mwh);

            }

        }

    '@

     

    #

    # Start notepad and get process object

    #

    $proc = [diagnostics.process]::start("notepad")

    start-sleep 1

    $mwh = $proc.MainWindowHandle

     

    #

    # Use window handle to attach to the main window

    # using the AutomationHelper class...

    #

    $rootElement = [AutomationHelper]::getroot($mwh)

     

    # Create condition to find the menu bar

    $menuBarFind = new-object Windows.Automation.PropertyCondition `

        ([Windows.Automation.AutomationElement]::AutomationIdProperty),"MenuBar"

     

    # Find the menu bar within the main window

    $menuBarElement = $rootElement.FindFirst("Children", $menuBarFind)

     

    # Create condition to find file menu

    $fileMenuFind = new-object Windows.Automation.PropertyCondition `

        ([Windows.Automation.AutomationElement]::AutomationIdProperty),"Item 1"

     

    # Find the File menu - this is where the script fails, because $menuBarElement is null

    $fileMenuElement = $menuBarElement.FindFirst(

        "Children", $fileMenuFind)

     

    # Now display the resulting prperty...

    [Console]::WriteLine($fileMenuElement.Current.AutomationId)

     

    ####################################################################

     

     

     

     

    Monday, August 06, 2007 12:14 AM
  • Has this been solved in V2?

    I ran Bruce's script with -sta switch, but it didn't work.

    Besides, the incline csharp code gave me such an error:
    The term 'csharp' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.

    Does anybody know how to make UIA work with Powershell? It's very good way to do GUI automation.
    Thursday, February 18, 2010 5:22 PM
  • I realized that if any UI item without classname could not be found with FindFirst or FindAll.
    You can test that by changing
    ([Windows.Automation.AutomationElement]::AutomationIdProperty), "Item 1"
    to
    ([Windows.Automation.AutomationElement]::AutomationIdProperty), "15" (I'm not sure whether this value is fixed everywhere. You may need to change it)
    which is the boby of notepad. Its classname is "Edit".
    Friday, February 19, 2010 6:03 AM
  • Hi Michael,

    I don't think my previous comment is the anwer. It's just a further anylasis.

    Do you know how to solve the problem? I mean to find an AutomationElement without ClassName.

    Thanks,

    9whirls
    Friday, February 19, 2010 4:54 PM
  • Hi, 9whirls.  Fair point - you're welcome to unmark as answer if you disagree. 

    But I'm not sure about a root workaround here.  I think searching for a null value is tricky -- it's hard for UIA to tell the difference between a classname of "" and a control that doesn't support the property at all.  I suspect that this might work erratically, since it's not really supported.

    If you can rephrase your search to seek something rather than nothing, I suspect it will go much better.

    If you really, really need to do what you are doing, don't use Find -- use a TreeWalker and do the check yourself.  TreeWalkers give much more flexibility, naturally.

    Thanks,
    Michael


    This posting is provided "AS IS" with no warranties, and confers no rights.
    Friday, February 26, 2010 5:28 PM
    Owner
  • Hi Michael,

    Thanks for your suggestion. However, treeWalker can't solve the problem. It can not retrive any UI element whose classname is "".
    The same program works well in C#.
    I highly suspect this is a bug of PowerShell.

    Thanks,
    9whirls
    Tuesday, March 02, 2010 7:15 AM