locked
XAML Benchmarking - need help for ARM RRS feed

  • Question

  • Hi, I'm trying to work out how my graphically intensive app will run on ARM, so need some help from someone(s) with an ARM tablet to run some benchmarking code and let me know the results...

    I've written a basic routine to move some sprites(images) around the screen, using rotation, translation, and opacity. There is a slider to change the number of objects on screen.

    On a Core2 E8400 with (crap) onboard graphics this will run smoothly (60FPS) with up to approx 400 objects on screen, at which point it appears to become GPU-bound.

    To run:

    • Create a new Blank VB Windows Store project in VS 2012.
    • Open MainPage.xaml.vb
    • Delete the template OnNavigatedTo method and replace with this:
    Protected Overrides Sub OnNavigatedTo(e As Navigation.NavigationEventArgs)
            Static Dim _rnd As New Random()
            Dim g As Grid = Me.Content
    
            Dim _actions As New List(Of Action)
    
            ' Make controls
            Dim slid As New Slider() With {.Orientation = Orientation.Vertical, .Height = 300, .Width = 50, .Maximum = 2000, .Minimum = 10, .HorizontalAlignment = Windows.UI.Xaml.HorizontalAlignment.Left, .VerticalAlignment = Windows.UI.Xaml.VerticalAlignment.Top, .Value = 50}
            g.Children.Add(slid)
    
            Dim regenSprites = Sub()
                                   ' Remove existing from playfield
                                   For Each anImg In g.Children.OfType(Of Image).ToList()
                                       g.Children.Remove(anImg)
                                   Next
                                   _actions.Clear()
    
                                   ' Add new
                                   For i = 1 To slid.Value
                                       Dim newSprite As New Image()
                                       newSprite.Stretch = Stretch.UniformToFill
                                       newSprite.Width = 70
                                       newSprite.Height = 70
                                       g.Children.Add(newSprite)
                                       newSprite.Source = New BitmapImage(New Uri("ms-appx:///Assets/Logo.png"))
                                       newSprite.HorizontalAlignment = Windows.UI.Xaml.HorizontalAlignment.Center
                                       newSprite.VerticalAlignment = Windows.UI.Xaml.VerticalAlignment.Center
                                       Dim trans As New CompositeTransform() With {.CenterX = newSprite.Width / 2, .CenterY = newSprite.Height / 2}
                                       newSprite.RenderTransform = trans
                                       Dim velocity As New Point(_rnd.NextDouble() * 6 - 3, _rnd.NextDouble() * 6 - 3)
                                       Dim rotSpeed = _rnd.NextDouble() * 4 - 2
                                       Dim opDelta = -(_rnd.NextDouble / 100)
                                       Dim currOp As Double = 1
    
                                       Dim sprUpdate = Sub()
                                                           Dim currX = trans.TranslateX
                                                           Dim currY = trans.TranslateY
                                                           Dim newX = currX + velocity.X
                                                           Dim newY = currY + velocity.Y
                                                           trans.TranslateX = newX
                                                           trans.TranslateY = newY
                                                           trans.Rotation += rotSpeed
    
                                                           currOp += opDelta
                                                           newSprite.Opacity = currOp
    
                                                           ' Check bounds and bounce if out
                                                           If Math.Abs(newX) > 350 Then
                                                               velocity = New Point(velocity.X * -1, velocity.Y)
                                                           End If
                                                           If Math.Abs(newY) > 350 Then
                                                               velocity = New Point(velocity.X, velocity.Y * -1)
                                                           End If
                                                           If currOp < 0 OrElse currOp > 1 Then
                                                               opDelta *= -1
                                                           End If
                                                       End Sub
    
    
                                       _actions.Add(sprUpdate)
                                   Next
                               End Sub
    
            regenSprites()
    
            Dim lblFPS As New TextBlock() With {.HorizontalAlignment = Windows.UI.Xaml.HorizontalAlignment.Center, .VerticalAlignment = Windows.UI.Xaml.VerticalAlignment.Top}
            g.Children.Add(lblFPS)
    
            Dim sw = Stopwatch.StartNew()
            Dim frameCnt = 0
            AddHandler CompositionTarget.Rendering, Sub()
                                                        For Each updFn In _actions
                                                            updFn()
                                                        Next
                                                        If sw.ElapsedMilliseconds >= 1000 Then
                                                            lblFPS.Text = "CPU FPS: " & frameCnt.ToString()
                                                            sw.Restart()
                                                            frameCnt = 0
                                                        End If
                                                        frameCnt += 1
                                                    End Sub
    
            AddHandler slid.ValueChanged, Sub(s, e2)
                                              regenSprites()
                                          End Sub
        End Sub
    

    Run that and observe some images bouncing around. Use the slider to change number of images.

    The CPU bound FPS is shown at the top of the screen. The CPU is used to generate the scene, I presume. If you have a newish machine you will struggle to get this less than 60, at which point you will have maxed out the UI thread.

    Depending on your machine, at some point the redraw will start to stutter. On my machine , this appears to be GPU bound at about 400 objects.

    What I want to know is:

    • On ARM, when does it start to stutter?
    • Is it CPU or GPU bound (i.e. when it starts to stutter is the shown FPS static or coming down?)

    If someone could attach an ARM device and run that and let me know results, I would be very grateful.

    Wednesday, November 28, 2012 8:54 PM

Answers

  • Your code sustains 60 fps with about 230 objects on my Surface.  With 400 objects, I get about 30 fps.

    • Proposed as answer by jrboddie Wednesday, December 5, 2012 4:37 AM
    • Marked as answer by LMKz Wednesday, December 5, 2012 5:08 AM
    Tuesday, December 4, 2012 10:16 PM

All replies

  • Hello,

    Unfortunately at this time we cannot benchmark developer's applications. As you can probably imagine, if we benchmarked yours we would likely see a flood of benchmark requests and people trying to make statements that appear to be "official" and from Microsoft. I really wish I could be of some assistance here but maybe someone in the community will be willing to go the extra mile for you.

    For more information on how your XAML app may perform in an ARM device check out this Channel 9 video: http://channel9.msdn.com/Events/Build/2012/4-103

    Best of luck with your app,

    James


    Windows SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/

    Monday, December 3, 2012 11:13 PM
    Moderator
  • Hello,

    Unfortunately at this time we cannot benchmark developer's applications. As you can probably imagine, if we benchmarked yours we would likely see a flood of benchmark requests and people trying to make statements that appear to be "official" and from Microsoft. I really wish I could be of some assistance here but maybe someone in the community will be willing to go the extra mile for you.

    For more information on how your XAML app may perform in an ARM device check out this Channel 9 video: http://channel9.msdn.com/Events/Build/2012/4-103

    Best of luck with your app,

    James


    Windows SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/

    Hi, I think you misunderstood my request; I'm not looking at getting my app benchmarked, I am just looking for some data that shows the relative performance of ARM to x86 when running managed XAML apps. One way to get some useful data would be to run my code on ARM and measure the performance.

    I think Microsoft have really let us down in that we are expected to write apps for ARM but there is no data available whatsoever that would give even a rough idea of HOW the apps will run. I shouldn't have to buy a Surface or equivalent just to test my apps.

    In my opinion, MS should either:

    • Produce an accurate ARM emulator running Windows 8 (as per the VS simulator).
    • or
    • Create a benchmark suite covering many different areas of performance (integer, FP, graphics etc) and release benchmark data for different hardware configurations, from lowest common denominator.

    At the moment, authoring for ARM is total guesswork. I don't even know whether my app is tested on ARM as part of the certification process...

    The link you provided is interesting, but doesn't solve the problem of estimating ARM performance. My app already runs very efficiently on x86, with minimal CPU and GPU use. But I don't know whether ARM will be 50% as fast or 5% (or 1%?).

    Lachlan


    Tuesday, December 4, 2012 9:50 PM
  • Your code sustains 60 fps with about 230 objects on my Surface.  With 400 objects, I get about 30 fps.

    • Proposed as answer by jrboddie Wednesday, December 5, 2012 4:37 AM
    • Marked as answer by LMKz Wednesday, December 5, 2012 5:08 AM
    Tuesday, December 4, 2012 10:16 PM
  • Your code sustains 60 fps with about 230 objects on my Surface.  With 400 objects, I get about 30 fps.

    Great stuff, thanks for that. So at 230 the animation is nice and smooth? And above that the displayed FPS on screen starts to decrease?

    That would tend to suggest that the limiting factor is CPU on the UI thread rather than the GPU, and that the CPU is quite a bit weaker than a Core 2 (as expected).

    Very useful, thanks.

    Lachlan

    Tuesday, December 4, 2012 11:04 PM
  • Yes, the animation is smooth but the sustained fps drops above 230 objects.  For object numbers above about 500, the slider becomes less responsive.

    Tuesday, December 4, 2012 11:29 PM
  • Follow up info for anyone else interested...

    You can add the following to your App.xaml.cs constructor to turn on built-in XAML debug info:

    DebugSettings.EnableFrameRateCounter = true

    This will show a few metrics, as described here: http://michaelcrump.net/enabling-frame-rate-counters-for-xaml-applications-in-windows-8

    The interesting one for the purposes of my example code is the right most, time spent in UI thread per frame. Once this gets above 16ms, you will drop under 60fps (1000/60 = 16.667..)

    If you are using Storyboard animations, it will also show CPU time in the compositor thread.

    Wednesday, December 5, 2012 3:32 AM