locked
How to add custom control to DirectX app in order to display touch keyboard

    Question

  • I have been reviewing several posts and see that in order to get the touch keyboard to display I need to create a custom control that supports the UI Automation TextPattern and ValuePatterns and focus changed events. XAML interop isnt an option here due to how far we are in development, an afternoons work into getting the app converted to be XAML/SwapChainBackgroundPanel based showed more issues than we would have time to overcome. We cant use anything Windows 8.1 specific either.

    How would I add this control to the CoreWindow? I dont believe I can use CreateWindow like I would in a normal application.

    Is there anything in particular that the control itself should derive from in?

    Wednesday, October 02, 2013 5:24 PM

Answers

  • Take a look at the UI Automation core window provider sample (Windows 8) for an example of how to implement and hook up an automation provider in a DirectX Windows Store app.

    "Control" is used loosely here: it just refers to whatever you have in your UI that the user can interact with. You will implement a COM object with the automation interfaces and return it in response to a CoreWindow::AutomationProviderRequested event.

    If you can I'd recommend interop with a Xaml text control though. Supplying the automation interfaces will trigger the keyboard, but you also need to implement the Text Services Framework (TSF) interfaces for non-keyboard input such as stylus, IME, and speech. TSF is much more complicated and we don't have any sample code for that.

    --Rob


    Wednesday, October 02, 2013 6:09 PM
    Owner
  • This is expected. The input pane doesn't communicate via UIA. It just uses UIA to know when a text is focused.

    For the basic keyboard you should be able to sync CoreWindow::CharacterReceived (and I think KeyDown, and KeyUp, but those aren't very useful for actual text input). For stylus input (and speech and IMEs) you will need to implement TSF. Again, I would strongly recommend interop with a Xaml TextBox for this.

    --Rob

    Thursday, October 03, 2013 11:44 PM
    Owner

All replies

  • Take a look at the UI Automation core window provider sample (Windows 8) for an example of how to implement and hook up an automation provider in a DirectX Windows Store app.

    "Control" is used loosely here: it just refers to whatever you have in your UI that the user can interact with. You will implement a COM object with the automation interfaces and return it in response to a CoreWindow::AutomationProviderRequested event.

    If you can I'd recommend interop with a Xaml text control though. Supplying the automation interfaces will trigger the keyboard, but you also need to implement the Text Services Framework (TSF) interfaces for non-keyboard input such as stylus, IME, and speech. TSF is much more complicated and we don't have any sample code for that.

    --Rob


    Wednesday, October 02, 2013 6:09 PM
    Owner
  • So I have merged some of the code from the UiaCoreWindowProvider sample over to a blank Direct3D windows store app. My control provider implements IValueProvider and ITextProvider, and in OnPointerPressed if the x is on the right side of the screen I raise a focus changed event which causes the keyboard to show up, which is great. If the x is on the left side of the screen I raise another focus changed event and have the AppWindowProvider return nullptr in GetFocus, which causes the keyboard to close, which is also great.

    The problem that is stumping me now is that none of the IValueProvider or ITextProvider methods I have implemented are being called, so I have no way to get the data being typed on the keyboard. Any thoughts on why this would be happening or what else I need to implement? I'm happy to provide my code so far if that would help figure out the issue.

    Thursday, October 03, 2013 11:39 PM
  • This is expected. The input pane doesn't communicate via UIA. It just uses UIA to know when a text is focused.

    For the basic keyboard you should be able to sync CoreWindow::CharacterReceived (and I think KeyDown, and KeyUp, but those aren't very useful for actual text input). For stylus input (and speech and IMEs) you will need to implement TSF. Again, I would strongly recommend interop with a Xaml TextBox for this.

    --Rob

    Thursday, October 03, 2013 11:44 PM
    Owner