XAML Benchmarking - need help for ARM

Question

Hi, I'm trying to work out how my graphically intensive app will run on ARM, so need some help from someone(s) with an ARM tablet to run some benchmarking code and let me know the results...

I've written a basic routine to move some sprites(images) around the screen, using rotation, translation, and opacity. There is a slider to change the number of objects on screen.

On a Core2 E8400 with (crap) onboard graphics this will run smoothly (60FPS) with up to approx 400 objects on screen, at which point it appears to become GPU-bound.

To run:

• Create a new Blank VB Windows Store project in VS 2012.
• Open MainPage.xaml.vb
• Delete the template OnNavigatedTo method and replace with this:

Protected Overrides Sub OnNavigatedTo(e As Navigation.NavigationEventArgs)
        Static Dim _rnd As New Random()
        Dim g As Grid = Me.Content

        Dim _actions As New List(Of Action)

        ' Make controls
        Dim slid As New Slider() With {.Orientation = Orientation.Vertical, .Height = 300, .Width = 50, .Maximum = 2000, .Minimum = 10, .HorizontalAlignment = Windows.UI.Xaml.HorizontalAlignment.Left, .VerticalAlignment = Windows.UI.Xaml.VerticalAlignment.Top, .Value = 50}
        g.Children.Add(slid)

        Dim regenSprites = Sub()
                               ' Remove existing from playfield
                               For Each anImg In g.Children.OfType(Of Image).ToList()
                                   g.Children.Remove(anImg)
                               Next
                               _actions.Clear()

                               ' Add new
                               For i = 1 To slid.Value
                                   Dim newSprite As New Image()
                                   newSprite.Stretch = Stretch.UniformToFill
                                   newSprite.Width = 70
                                   newSprite.Height = 70
                                   g.Children.Add(newSprite)
                                   newSprite.Source = New BitmapImage(New Uri("ms-appx:///Assets/Logo.png"))
                                   newSprite.HorizontalAlignment = Windows.UI.Xaml.HorizontalAlignment.Center
                                   newSprite.VerticalAlignment = Windows.UI.Xaml.VerticalAlignment.Center
                                   Dim trans As New CompositeTransform() With {.CenterX = newSprite.Width / 2, .CenterY = newSprite.Height / 2}
                                   newSprite.RenderTransform = trans
                                   Dim velocity As New Point(_rnd.NextDouble() * 6 - 3, _rnd.NextDouble() * 6 - 3)
                                   Dim rotSpeed = _rnd.NextDouble() * 4 - 2
                                   Dim opDelta = -(_rnd.NextDouble / 100)
                                   Dim currOp As Double = 1

                                   Dim sprUpdate = Sub()
                                                       Dim currX = trans.TranslateX
                                                       Dim currY = trans.TranslateY
                                                       Dim newX = currX + velocity.X
                                                       Dim newY = currY + velocity.Y
                                                       trans.TranslateX = newX
                                                       trans.TranslateY = newY
                                                       trans.Rotation += rotSpeed

                                                       currOp += opDelta
                                                       newSprite.Opacity = currOp

                                                       ' Check bounds and bounce if out
                                                       If Math.Abs(newX) > 350 Then
                                                           velocity = New Point(velocity.X * -1, velocity.Y)
                                                       End If
                                                       If Math.Abs(newY) > 350 Then
                                                           velocity = New Point(velocity.X, velocity.Y * -1)
                                                       End If
                                                       If currOp < 0 OrElse currOp > 1 Then
                                                           opDelta *= -1
                                                       End If
                                                   End Sub


                                   _actions.Add(sprUpdate)
                               Next
                           End Sub

        regenSprites()

        Dim lblFPS As New TextBlock() With {.HorizontalAlignment = Windows.UI.Xaml.HorizontalAlignment.Center, .VerticalAlignment = Windows.UI.Xaml.VerticalAlignment.Top}
        g.Children.Add(lblFPS)

        Dim sw = Stopwatch.StartNew()
        Dim frameCnt = 0
        AddHandler CompositionTarget.Rendering, Sub()
                                                    For Each updFn In _actions
                                                        updFn()
                                                    Next
                                                    If sw.ElapsedMilliseconds >= 1000 Then
                                                        lblFPS.Text = "CPU FPS: " & frameCnt.ToString()
                                                        sw.Restart()
                                                        frameCnt = 0
                                                    End If
                                                    frameCnt += 1
                                                End Sub

        AddHandler slid.ValueChanged, Sub(s, e2)
                                          regenSprites()
                                      End Sub
    End Sub

Run that and observe some images bouncing around. Use the slider to change number of images.

The CPU bound FPS is shown at the top of the screen. The CPU is used to generate the scene, I presume. If you have a newish machine you will struggle to get this less than 60, at which point you will have maxed out the UI thread.

Depending on your machine, at some point the redraw will start to stutter. On my machine , this appears to be GPU bound at about 400 objects.

What I want to know is:

• On ARM, when does it start to stutter?
• Is it CPU or GPU bound (i.e. when it starts to stutter is the shown FPS static or coming down?)

If someone could attach an ARM device and run that and let me know results, I would be very grateful.

Answer 1

Hello,

Unfortunately at this time we cannot benchmark developer's applications. As you can probably imagine, if we benchmarked yours we would likely see a flood of benchmark requests and people trying to make statements that appear to be "official" and from Microsoft. I really wish I could be of some assistance here but maybe someone in the community will be willing to go the extra mile for you.

For more information on how your XAML app may perform in an ARM device check out this Channel 9 video: http://channel9.msdn.com/Events/Build/2012/4-103

Best of luck with your app,

James

---

Windows SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/

Answer 2

Hello,

Unfortunately at this time we cannot benchmark developer's applications. As you can probably imagine, if we benchmarked yours we would likely see a flood of benchmark requests and people trying to make statements that appear to be "official" and from Microsoft. I really wish I could be of some assistance here but maybe someone in the community will be willing to go the extra mile for you.

For more information on how your XAML app may perform in an ARM device check out this Channel 9 video: http://channel9.msdn.com/Events/Build/2012/4-103

Best of luck with your app,

James

---

Windows SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/

Hi, I think you misunderstood my request; I'm not looking at getting my app benchmarked, I am just looking for some data that shows the relative performance of ARM to x86 when running managed XAML apps. One way to get some useful data would be to run my code on ARM and measure the performance.

I think Microsoft have really let us down in that we are expected to write apps for ARM but there is no data available whatsoever that would give even a rough idea of HOW the apps will run. I shouldn't have to buy a Surface or equivalent just to test my apps.

In my opinion, MS should either:

• Produce an accurate ARM emulator running Windows 8 (as per the VS simulator).
• or
• Create a benchmark suite covering many different areas of performance (integer, FP, graphics etc) and release benchmark data for different hardware configurations, from lowest common denominator.

At the moment, authoring for ARM is total guesswork. I don't even know whether my app is tested on ARM as part of the certification process...

The link you provided is interesting, but doesn't solve the problem of estimating ARM performance. My app already runs very efficiently on x86, with minimal CPU and GPU use. But I don't know whether ARM will be 50% as fast or 5% (or 1%?).

Lachlan

Answer 3

Your code sustains 60 fps with about 230 objects on my Surface.  With 400 objects, I get about 30 fps.

Answer 4

Your code sustains 60 fps with about 230 objects on my Surface.  With 400 objects, I get about 30 fps.

Great stuff, thanks for that. So at 230 the animation is nice and smooth? And above that the displayed FPS on screen starts to decrease?

That would tend to suggest that the limiting factor is CPU on the UI thread rather than the GPU, and that the CPU is quite a bit weaker than a Core 2 (as expected).

Very useful, thanks.

Lachlan

Answer 5

Yes, the animation is smooth but the sustained fps drops above 230 objects.  For object numbers above about 500, the slider becomes less responsive.

Answer 6

Follow up info for anyone else interested...

You can add the following to your App.xaml.cs constructor to turn on built-in XAML debug info:

DebugSettings.EnableFrameRateCounter = true

This will show a few metrics, as described here: http://michaelcrump.net/enabling-frame-rate-counters-for-xaml-applications-in-windows-8

The interesting one for the purposes of my example code is the right most, time spent in UI thread per frame. Once this gets above 16ms, you will drop under 60fps (1000/60 = 16.667..)

If you are using Storyboard animations, it will also show CPU time in the compositor thread.