locked
Latency in XAML based Metro apps RRS feed

  • Question

  • Hi, we're seeing a strange sluggishness in XAML based Metro apps that we don't see when using HTML5 Metro apps.  We noticed this first when drawing ink - that with HTML5 apps (like the InkPad sample) the ink was drawn very close to where the tip of the stylus was on the screen.  But when attempting the same thing on a XAML canvas using a Path object to draw the ink, the ink lagged behind the stylus by a noticeable amount.

    Once we noticed it, we see this lag in other places as well.  The simplest example I have is to create a large canvas (or one that covers the entire screen), and add a rectangle to it.  Implement events on the rectangle for PointerMoved, PointerPressed, and PointerReleased, and manipulate Canvas.Get/SetLeft/Top so that the rectangle is moved when you drag it (see source below).  When you run this app, the rectangle lags noticeably behind the pointer.  I see the same problem in both C++ and C#.

    This is more than the "typical" touch pointer lag.  As an example, in Windows desktop with a stylus, grab a window and move it around the screen.  The window will fall behind the stylus a little bit.  But the window always keeps up with the stylus "cross-hairs" - in other words, the cross-hairs will fall slightly behind the stylus, but the window is fixed relative to the cross-hairs.  In the XAML example above, you see not only the normal stylus lag, but the rectangle will fall behind the cross-hairs as well, and the combination of the two leads to a significant and annoying latency in the app.  In fact with a mouse the PointerMoved events are often lost because the rectangle falls too far behind the mouse.

    I had been hoping this problem would be cleared up in the Consumer Preview but it is still there.  Is this a bug, or do I need to do something special to eliminate this latency?  Thanks for your help.

    namespace Application1
    {
        /// <summary>
        /// An empty page that can be used on its own or navigated to within a Frame.
        /// </summary>
        public sealed partial class BlankPage : Page
        {
            public BlankPage()
            {
                this.InitializeComponent();
                m_deltaX = 0;
                m_deltaY = 0;
                m_shapeSelected = null;
            }
    
            /// <summary>
            /// Invoked when this page is about to be displayed in a Frame.
            /// </summary>
            /// <param name="e">Event data that describes how this page was reached.  The Parameter
            /// property is typically used to configure the page.</param>
            protected override void OnNavigatedTo(NavigationEventArgs e)
            {
            }
    
            private void ShapeMove(object sender, PointerEventArgs e)
            {
                if (e.Pointer.IsInContact && m_shapeSelected != null)
                {
                    Windows.UI.Input.PointerPoint ptPt = e.GetCurrentPoint(m_canvas);
                    Windows.Foundation.Point pt = ptPt.Position;
                    Canvas.SetLeft(m_shapeSelected, pt.X - m_deltaX);
                    Canvas.SetTop(m_shapeSelected, pt.Y - m_deltaY);
                }
            }
            private void ShapePressed(object sender, PointerEventArgs e)
            {
                m_shapeSelected =  sender as Windows.UI.Xaml.UIElement;
                if (m_shapeSelected != null)
                {
                    Windows.UI.Input.PointerPoint ptPt = e.GetCurrentPoint(m_canvas);
                    Windows.Foundation.Point pt = ptPt.Position;
                    double left = Canvas.GetLeft(m_shapeSelected);
                    double top = Canvas.GetTop(m_shapeSelected);
                    m_deltaX = pt.X - left;
                    m_deltaY = pt.Y - top;
                }
            }
            private void ShapeReleased(object sender, PointerEventArgs e)
            {
                m_shapeSelected = null;
            }
            private Windows.UI.Xaml.UIElement m_shapeSelected=null;
            private double m_deltaX;
            private double m_deltaY;
        }
    }
    

    Thursday, March 1, 2012 10:36 PM

Answers

  • Hi Derek,

    Moving a rectangle across a canvas using Pointer events in conjunction with setting Canvas.Top and Left properties in every PointerMoved event is bound to cause some latency since the individual dependency properties getting updated will hold up the raw pointer processing in the UI thread. I would recommend using TIE's manipulation events in conjunction with using RenderTransform for this scenario as illustrated in the XAML Input SDK sample (Scenario 4). This latter approach will reduce latency to a large extent.

    In order to be completely non-latent, the input needs to be processed on a separate thread from the UI thread and one of the methods to do this will be to use the SwapChainBackgroundPanel approach. We recommend using this approach for inking scenarios in XAML in Windows 8.

    Thanks!


    Harini Kannan | Program Manager, XAML

    Monday, December 3, 2012 6:01 PM

All replies

  • Can you send the application?  MSMALL at Microsoft.

    Matt Small - Microsoft Escalation Engineer - Forum Moderator

    Friday, March 2, 2012 3:59 PM
    Moderator
  • currently investigating this and will update the post when we know more.

    Matt Small - Microsoft Escalation Engineer - Forum Moderator

    Wednesday, March 7, 2012 7:33 PM
    Moderator
  • We are still investigating this. I will update again next week.


    Matt Small - Microsoft Escalation Engineer - Forum Moderator

    Tuesday, March 13, 2012 2:13 PM
    Moderator
  • I have an official response from the product team:

    The recommended approach for interacting with elements through touch interactions is by placing them inside of a ScrollViewer.  This will allow them to be panned and zoomed with the lowest latency and best frame rate that is possible in XAML applications.  There is also a SwapChainBackgroundPanel API to enable DirectX interop for more complicated gaming scenarios.  The XAML team is actively pursuing performance improvements in all scenarios, and will continue to look into touch latency issues as they arise.

    The separate issue of pointer events being lost is likely the result of not capturing the pointer.  The CapturePointer and ReleaseCapturePointer methods allow a UIElement to receive all incoming pointer events for a particular time period.  This would prevent an element from losing events that occur on locations outside of its bounds.


    Matt Small - Microsoft Escalation Engineer - Forum Moderator

    Thursday, March 15, 2012 1:46 PM
    Moderator
  • We're talking about moving a box across a canvas.  I don't think that's entered the realm of "complex gaming scenarios" since 1987.  And surely you can appreciate the absurdity of suggesting a ScrollViewer as a solution.  Would you change the size of an object in Visio by zooming in on it? 

    Remember - the original concern was that when drawing ink, the ink lags behind the stylus significantly more than it does using HTML5.  And the recommendation is to use DirectX???  It sounds like this concern has not been taken seriously.

    Thursday, March 15, 2012 4:34 PM
  • Hi Derek - Rob Caplan is going to chime in with more information than I've given you from the product group.  Please know that your concerns are definitely being heard.


    Matt Small - Microsoft Escalation Engineer - Forum Moderator


    Thursday, March 15, 2012 5:47 PM
    Moderator
  • Hi Derek,

    Apologies for the confusion here. As you point out, solutions which may be viable for your sample case (moving an existing around) aren't applicable to the inking case (where new elements need to be created). I've been discussing this with both the Xaml and Inking teams and we'll follow up with you once we have something we can report.

    --Rob

    Friday, March 16, 2012 2:00 AM
    Moderator
  • I'm seeing similar issue with DirectX + XAML interop apps as well, it seems that the pointers events are lagging when XAML is used. Derek do you know if this issue is solved in the release preview?
    Friday, June 22, 2012 4:41 AM
  • I'm seeing similar issue with DirectX + XAML interop apps as well, it seems that the pointers events are lagging when XAML is used.

    Same problem here. It seems that there are too many input events and message queue fills up very quickly.
    Friday, July 20, 2012 7:53 AM
  • I've been seeing the same issue. The only way I've gotten rid of the latency in a Metro app is to use DirectX without XAML.
    Tuesday, September 4, 2012 10:18 PM
  • I've got the same problem in our XAML+DirectX projects.
    Friday, October 19, 2012 4:30 PM
  • Though this isn't a solution, here's a way how you could improve performance a bit when touch lag is observed:

    Put this before window initialization

    using namespace Windows::UI::Input;
    
    ....
    
    	PointerVisualizationSettings^ pointerVisualizationSettings = PointerVisualizationSettings::GetForCurrentView();
    	pointerVisualizationSettings->IsContactFeedbackEnabled = false;
    	pointerVisualizationSettings->IsBarrelButtonFeedbackEnabled = false;

    I've posted the code in C++, but it's trivial to convert to C#, remove hats, and change '->' to '.'

    This disables omni-transparent graphical path which is displayed when you're moving finger around the screen.

    I am guessing Mayamba is right, and message queue gets filled up very quickly when you touch the screen.


    • Edited by Tomas Dirvanauskas Thursday, November 22, 2012 4:54 PM Code posted in C++, but people were probably expecting C#
    • Proposed as answer by lkm_km Friday, March 8, 2013 7:51 PM
    Thursday, November 22, 2012 4:51 PM
  • @ Rob Caplan

    Have you found a solution during the last 8 months?

    I find this issue very important for the acceptance of Windows 8.

    Thursday, November 22, 2012 8:35 PM
  • Just tried out the new Visual Studio 2012 Update 1, and sadly the issue is still there :(
    Wednesday, November 28, 2012 8:56 PM
  • Hi Derek,

    Moving a rectangle across a canvas using Pointer events in conjunction with setting Canvas.Top and Left properties in every PointerMoved event is bound to cause some latency since the individual dependency properties getting updated will hold up the raw pointer processing in the UI thread. I would recommend using TIE's manipulation events in conjunction with using RenderTransform for this scenario as illustrated in the XAML Input SDK sample (Scenario 4). This latter approach will reduce latency to a large extent.

    In order to be completely non-latent, the input needs to be processed on a separate thread from the UI thread and one of the methods to do this will be to use the SwapChainBackgroundPanel approach. We recommend using this approach for inking scenarios in XAML in Windows 8.

    Thanks!


    Harini Kannan | Program Manager, XAML

    Monday, December 3, 2012 6:01 PM
  • Hi Harini,

    I am happy that a Microsoft employee is participating again in this discussion! Thank you!

    What you wrote increases performance. But I do use DirectX and SwapChainBackgroundPanel already, and still there is a lag when using touch input (in contrast to mouse input, which is without delay).

    You can see it in the Microsoft example "DirectX postcard app sample".

    In this sample app, enter text and move it around. If you use your mouse, there is no delay. Use your finger, the text will always follow your finger with a delay.

    Wednesday, December 5, 2012 7:40 AM
  • @Tomas

    Thanks a lot, this solved all of my issues. Microsoft should give a hint to developers on their resources that the small path behind the finger causes such troubles.
    My apps frames per second went fromt 60 to 30 when I put 5 fingers on the screen. Now it is stable 60 fps.

    Friday, March 8, 2013 7:52 PM