none
SL5 - immediate mode 2d API

    Question

  • Hi

    in SL5 press-release notes there is such a sentence
    "GPU-accelerated XNA-compatible 3D and immediate-mode 2D API

    did anyone see some sample of immediate mode 2D API? At the MIX they showed some Hello World samples for 3D API and nothing more.

    What classes do let you work with 2D API in SL5 beta? I could not find them?
    Thx 

    Tuesday, June 28, 2011 1:02 AM

Answers

  • Ah, nice example, thanks.

    Here are some things to consider.

    • Brush updates are expensive because it causes the element to be rasterized again (even if you're caching it). Try not to change those on an element once set.
    • The more elements in your visual tree, the slower things are going to get because they get processed during a draw. Bitmap caching is one way to reduce the elements being processed each frame since a cache on a parent container essentially collapses the tree. Manually rendering to a WriteableBitmap and then moving that bitmap around can also work.
    • Don't set a cache on a parent panel and then move the elements inside it every frame since this invalidates the cached image of the panel. Enabling caching at the panel level is best for static child content (like your chart). For dynamic content, the parent shouldn't be bitmap cached but the moving content should (like the sine wave sample).
    • Since your chart is static content once generated, if you break your scene into pre-rendered tiles (or arrange your bitmap cache this way) and just scroll those tiles instead of updating each element on the fly you should get really good performance.
    • You might consider replacing the many green lines in that second pane to be a filled path more like what's in the third pane. On my screen those lines are very dense and close together, so visually I think it would look about the same, but you would cut down on some of the per frame processing. You know your app and customers best, so that may not be an acceptable optimization. If you need the lines, group those into panels and turn bitmap caching on for the panel. You could then compose and scroll those panels at high speeds since there would be no additional CPU rasterization and the composition would happen on the GPU.
    • I would suggest a similar optimization for your primary pane. Your scenario here is ideal for what we call a "virtualizing canvas". Basically, you have more data than you need to show on the screen at one time. There are various strategies for how to deal with this. You could break your data into a sequence of say 12 or so panels, turn on bitmap caching for the panels, and then add only 2 of the panels to the visual tree at a time, swapping in and out the active panels as the user scrolls. You can build a custom canvas control to handle that virtualization. It's possible VirtualizingPanel can provide a foundation to help with this, but I haven't used it myself.
    Wednesday, June 29, 2011 4:15 PM

All replies

  • Looks like a typo, I'll see about getting that updated.

    You can do 2D using the 3D APIs, but you have to rasterize yourself. Alternatively you can continue using the existing 2D shape APIs in Silverlight.

    What are you looking to build with 2D immediate mode APIs and what features would help?

    Tuesday, June 28, 2011 4:37 PM
  • Expecting to see an API like GDi+ that would work fast , very fast, not like current snail-shapes. Something like DrawingContext in WPF. I know I'm dreaming big, was bugging S. Guthrie for last 3 years, but I see this won't happen in near future. 
    I need that to be able paint fast bunch of shapes the way I need, not they way I'm pushed by using Shapes or Path.

    At least StreamingGeometry would be good to have.

    Please update me with something good.

    Thx 

    Tuesday, June 28, 2011 4:48 PM
  • Thanks for the details. What type of app would this be for? Game? Data visualization?

    One of the goals for the 3D APIs was to open up the platform to those kinds of scenarios. It's possible to implement the APIs you described on top of the 3D functionality provided in SL5. We couldn't build out this higher-level functionality this release, but it should be possible from a technical standpoint for developers with the inclination to port an external rasterizer over and expose it via 2D direct drawing APIs themselves.

    Tuesday, June 28, 2011 5:02 PM
  • Data visualiation - stock charts where we need to display bunch of rectangle and update real time. The only one way to achieve is to use WriteableBitmap. But I was hoping MS would provide a more high-level metods for this, now we need to paint everything by hand, pixel-by-pixel. Not imho the best supprot for loyal developers from MS side.

    Tuesday, June 28, 2011 5:13 PM
  • If the performance of WriteableBitmap is acceptable for you, no need to write individual pixels yourself. You can use a project like this one:

    http://writeablebitmapex.codeplex.com/

    Such a library could also be ported on top of the 3D API.

    Alternatively, since you primarily need to draw rectangles, that's one of the simpler things to do in 3D and you could adapt this sample:

    http://code.msdn.microsoft.com/Colored-Non-Textured-3D-c0e3cda6

    Or create a class similar to the following:

    http://msdn.microsoft.com/en-us/library/microsoft.xna.framework.graphics.spritebatch.aspx

    Tuesday, June 28, 2011 5:44 PM
  • well, of course I know about that library, I even contributed to it to be able paint anti-aliased lines. But laks many more features, say gradient, or cliping.

    Well, using 3D stuff for 2D is a little of overkill.

    Why can't we have the simple DrawingContext or StreamingGeometry from WPF? that would halp bunch of people on this forum. 

    Just trying to keep things simple.

    Tuesday, June 28, 2011 5:53 PM
  • You can have an API like that, just at the moment it's not built in. :-)

    The APIs you mentioned are internally implemented on 3D since that's how one communicates with graphics cards these days. Now that we have provided a 3D API it is not only possible for us to implement this, but for external developers as well.

    I understand you would have liked us to provide 2D direct drawing features out of the box this release to simplify development for your scenario, but unfortunately that wasn't possible. We're tracking the feature request.

    In the meantime, this looks like a great opportunity for a community project to build a library.

    Tuesday, June 28, 2011 6:15 PM
  • ok, thx
    problem is that I don't have enogh knowledge  about 3D (OpenGL mostly) stuff to create a nice 2D library using your 3D. And not sure when someone will build one especially when Internet is full of rumors about Silverlight being dumped and replaced by html5 in Windows 8.

    It looks like I'll have to wait for another 2-3 years or maybe change to html5 and canvas, at least there we have somekind of 2D api that works fast enough. 

    Tuesday, June 28, 2011 6:24 PM
  • If you would like to share a little more about how you were using the shape APIs we might be able to get to the bottom of the performance issue you ran into. Any chance you can put a repro app online for us to take a look at?

    With cached composition you should be able to create and scale rectangles in hardware at good speeds.

    Tuesday, June 28, 2011 6:55 PM
  • well, actually quite simple. 
    paint 2000 rectangles with random gradient background using sin(x) formula, use a random height for each.

    now when resizing the browser window adjust those rectangles width to fill up whole available width of parent canvas. you'll notice the slowness of this process. 
    you can't use Path here, cause we have random gradient background, so you'll need to create a Path for each rectangle or create a rectanlge object  itself. Either way is slow. 

    if you create a project that show how to do it right I'm pretty sure a lot of people will be happy.


    Tuesday, June 28, 2011 7:11 PM
  • 2000 rectangles sounds like a dense chart! ;-)

    To limit other noise in the test, we should take browser resize out of the scenario and focus on the scaling rectangle drawing you're interested in.

    I'll check into whether we have a comparable sample and if not see what we can do to publish one.

    Wednesday, June 29, 2011 12:51 AM
  • he he, tell me about dense charts, we have clients that wanna see 50000 rectangles :) but for this scenario 2000 would suffice .

    ok, show me what you've got.

    Charlez Petzold has some nice samples on how to paint on DrawingVisual big amounts of shapes, but again, in Silverlight that's not possible.

    Wednesday, June 29, 2011 12:56 AM
  • just to prove that words are correct about immediate mode 2D api, here is a screenshot from your web-site. 

    Wednesday, June 29, 2011 1:00 AM
  • Thanks, yes, it was a typo. Someone should be correcting that soon.

    Here is a sample project drawing 2000 gradient filled rectangles to animate 4 sine waves @ 60fps on my laptop.

    https://skydrive.live.com/?cid=d0aac0a96dc0e888&sc=documents&uc=1&id=D0AAC0A96DC0E888%21169

    You will need Silverlight 5 installed since it demonstrates the new independent animation features. Make sure you run it using the supplied HTML or web project since the page is configured to enable GPU acceleration.

    I have a newer build, but this should work on the beta.

    Hope this helps. Let me know if I can answer any questions on the code.

    Wednesday, June 29, 2011 3:39 AM
  • couple of issues.

    1. it looks like you used a newer version that is not available to the public, so SL runtime tries to update itself, when opening the update page I get a message that this version is not yet available.

    2. I'm testing this in a virtual PC, and if I have 
    <param name="EnableGPUAcceleration" value="true" /> 

    then IE fails to load, if I set that to false IE loads fine, but then again I have that message about upgrade.

    Can I get that new version that you are using?

    Is "BitmapCache" something new in SL5?

    Wednesday, June 29, 2011 4:26 AM
  • Try updating the minRuntimeVersion in the HTML to match the beta -- "5.0.60211.0" should work. It's possible most of the gains are coming from enabling bitmap caching on the elements, so SL4 might give decent results too, but I didn't test.

    Many VM video drivers don't allow full access to the GPU and instead emulate parts in software (which defeats any of the GPU performance gains). You're probably going to have to run this on real hardware.

    BitmapCache indicates the element should be cached to a texture and composed on the GPU. It has been available since SL3. This page here discusses it briefly.

    New for SL5 is that we run animations for cached elements from a different thread than the application thread and so they animate "independently" -- net result is smoother animation.


    Wednesday, June 29, 2011 11:50 AM
  • if I install SL5 beta on my main PC, do you guarantee that old SL4 projects will work as well? :) And VS 2010 Ultimate won't get crazy cause of SL5 beta? 

    Wednesday, June 29, 2011 1:11 PM
  • Depends on the project. As it is a beta, there are some temporary behavior changes in the runtime to be resolved by RTM. I have had pretty good luck uninstalling and reinstalling the SL toolchain as needed, but your mileage may vary.

    You could also try running the project on SL4 first to see if bitmap caching alone is enough of a boost.

    Wednesday, June 29, 2011 1:25 PM
  • btw, after I install SL 5 beta tools I'm not able to create a new project in VS2010 that woudl target SL5. I'd have to change by hand the project file. Any idea how this can be fixed?

    Wednesday, June 29, 2011 2:21 PM
  • ok, I tried on main PC. 

    Try to resize the browser, that's where the slowness starts to be seen. Comment animation, I don't need it, but make it paint fast on browser resize.

    That's a way to see how fast the actual paint is.

    In my case I need also to scroll the rectangles left & right, and in such scenario the caching doesn't really help and everything is very slow. However it works just fine in WPF. (I'm sharing same code for math)

    Wednesday, June 29, 2011 2:46 PM
  • Hi there. Just adding my five cents here, maybe it helps. A while ago I've had a discussion with a forum member about the missing point sprites in XNA 4.0. In the end the solution was to create a vertex shader that handles the pixel perfect rendering, and performance was quite amazing - in my tests I rendered ~700,000 points (each consisting of two triangles) with 60 fps. The discussion can be found here. It also contains links to a sample implementation for a SpriteBatch replacement in SL5, and links to the final result with source code.

    I know your requirements are a bit different, and this doesn't solve the initial issue that we don't have a simple API for things like that. However, I just wanted to say that it looks much harder to use 3D for 2D content than it actually is, and the resulting performance is outstanding. And of course, the available features of the 3D API outrun what you can do with WriteableBitmap(Ex) by far.


    Wednesday, June 29, 2011 2:54 PM
  • RE: VS and SL5 - What's the link to the package you installed? Make sure it was the tools package and not just the SDK.

    RE: Resize - That's not a good test for raw drawing speed of the shape APIs since browser resize causes layout to run and potentially for resources to be released and re-created.

    RE: Scrolling - Not sure exactly what you need there, but you can scroll the rectangles left and right with another transform or by translating their parent.

    Wednesday, June 29, 2011 2:57 PM
  • RE: VS and SL5 - got it from here http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=23887

    RE: Resize - and what shall I in this case? Prohibit user resize window?

    RE: Scrolling - in my case I have visible about 400 rectangles. when user scrolls my chart left & right, I do reuse the visible rectangles and just change their Left, Top and Brush properties by some internal logic I have. What I noticed from profiler that setting Brush property is the most expensive operation. 

    Wednesday, June 29, 2011 3:02 PM
  • just remembered, another big issue there is in Silverlight, there isn't a fast way to get the dimensions of a given text.
    That's what I use in WPF

        public Size GetTextMetrics(string text, string fontName, double fontSize)
        {
          Typeface typeface = new Typeface(fontName);
          FormattedText f = new FormattedText(text,
                                              CultureInfo.CurrentUICulture,
                                              FlowDirection.LeftToRight,
                                              typeface,
                                              fontSize,
                                              null);
          return new Size(f.WidthIncludingTrailingWhitespace, f.Height);
        }

        public Size GetTextMetrics(string text, string fontName, double fontSize)

        {

          Typeface typeface = new Typeface(fontName);

          FormattedText f = new FormattedText(text,

                                              CultureInfo.CurrentUICulture,

                                              FlowDirection.LeftToRight,

                                              typeface,

                                              fontSize,

                                              null);


          return new Size(f.WidthIncludingTrailingWhitespace, f.Height);

        }

    in SL I have to simulate with a hidden TextBlock which is very slow. Any better idea to have such a code in SL? Wuld SL5 bring something better from this point of view?

    Thx for your help! 

    Wednesday, June 29, 2011 3:12 PM
  • RE: Resize - It's been my experience that most users are fairly tolerant of a small delay after a resize since it's generally an infrequent operation. My point was regarding drawing throughput in the normal case. Not saying there isn't something worth looking at there, just that I hope browser resize speed is not what most would consider an adoption blocker.

    RE: Scrolling - Does your chart remain static once drawn? If so, you could attach them to a parent Canvas, turn on BitmapCache for the Canvas itself, then scroll that Canvas instead of each individual rect. This can give you the performance of just scrolling a single image instead of a lot of little rectangles.

    @Peter - Nice example! Cached composition works great in many cases, but the 3D APIs are definitely the most scalable approach.

    Wednesday, June 29, 2011 3:16 PM
  • Re: Resize - ok we can live with that
    RE: Scrolling - yes, after that chart is static, or in some case only last rectangle or another small part is updated. But not sure I'm getting correct your idea.
    Say I have 5000 data records. Do you propose me to create 5000 rectangles and place them on canvas, and after that just scroll the canvas to needed position?
    What about when user zoom in & out the number of visible bars. How can I proceed here?
    you can see a loive sample of the chart here

    http://platform.modulusfe.com/stockchartxsl.rtw/TestChartTestPage.html 


    you can see that after you zoom to max allowed range the chart scrolling ir pretty slow.

    Wednesday, June 29, 2011 3:30 PM
  • Ah, nice example, thanks.

    Here are some things to consider.

    • Brush updates are expensive because it causes the element to be rasterized again (even if you're caching it). Try not to change those on an element once set.
    • The more elements in your visual tree, the slower things are going to get because they get processed during a draw. Bitmap caching is one way to reduce the elements being processed each frame since a cache on a parent container essentially collapses the tree. Manually rendering to a WriteableBitmap and then moving that bitmap around can also work.
    • Don't set a cache on a parent panel and then move the elements inside it every frame since this invalidates the cached image of the panel. Enabling caching at the panel level is best for static child content (like your chart). For dynamic content, the parent shouldn't be bitmap cached but the moving content should (like the sine wave sample).
    • Since your chart is static content once generated, if you break your scene into pre-rendered tiles (or arrange your bitmap cache this way) and just scroll those tiles instead of updating each element on the fly you should get really good performance.
    • You might consider replacing the many green lines in that second pane to be a filled path more like what's in the third pane. On my screen those lines are very dense and close together, so visually I think it would look about the same, but you would cut down on some of the per frame processing. You know your app and customers best, so that may not be an acceptable optimization. If you need the lines, group those into panels and turn bitmap caching on for the panel. You could then compose and scroll those panels at high speeds since there would be no additional CPU rasterization and the composition would happen on the GPU.
    • I would suggest a similar optimization for your primary pane. Your scenario here is ideal for what we call a "virtualizing canvas". Basically, you have more data than you need to show on the screen at one time. There are various strategies for how to deal with this. You could break your data into a sequence of say 12 or so panels, turn on bitmap caching for the panels, and then add only 2 of the panels to the visual tree at a time, swapping in and out the active panels as the user scrolls. You can build a custom canvas control to handle that virtualization. It's possible VirtualizingPanel can provide a foundation to help with this, but I haven't used it myself.
    Wednesday, June 29, 2011 4:15 PM
  • 2000 rectangles sounds like a dense chart! ;-)

    Aaron, drawing hundreds of thousands or even millions of points in realtime is what we often do in exploratory data mining. Two thousand rectangles is nothing.

    In that respect, Silverlight is a huge disappointment... fortunately, there is WriteableBitmap, but it feels like I'm back to 90s writing directly to video buffer (A000h, anyone? :))

    Tuesday, November 01, 2011 10:21 PM
  • That's taking my statement a little out of context, or certainly its intent. A data mining scenario with extreme graphics requirements will need a different solution. The 3D APIs which allow you to offload more work to the GPU might be a better fit.

    http://msdn.microsoft.com/en-us/library/gg197424(v=xnagamestudio.35).aspx

    Wednesday, November 02, 2011 12:34 PM
  • Could you use CentauriBoy's BasicPrimitives (2d xna library) for optimal pref.

    http://www.varcade.com/blog/?p=147

    Tuesday, November 29, 2011 9:14 AM
  • Hello

    Here you can find the code snippet for #D application

    http://msdn.microsoft.com/en-us/library/gg197425%28v=XNAGameStudio.35%29.aspx

    and overview about class and library @ below

    http://msdn.microsoft.com/en-us/library/gg197424%28v=XNAGameStudio.35%29.aspx

    Hope it helps..

    and you can get code from here also.

    http://www.codeproject.com/KB/silverlight/silverlight-5.aspx

    Thursday, December 08, 2011 1:22 AM
  • Yeah, and you could "use a sledge-hammer to crack nuts". Wink

    XNA is not meant to paint financial charts and diagrams, that should be supported by SL itself, as it was in Windows by having simple and fast painting methods, even WPF is slow when it comes about rendering complex financial charts, I still have to use old WinAPI methods to paint all my charts.

    Thursday, December 08, 2011 2:02 AM
  • Hey EugenUS, did you find a solution for your Silverlight stock chart? I saw an earlier post mentioned WriteableBitmapEx (which you have contributed to, and so have I!). I am about to release a component called SciChart, which is a fast WPF / Silverilght stock chart component. It uses this technology to draw large datasets (~100,000 OHLC datapoints). Please see the performance demos of SciChart here.

    I built this as I have scoured the web for a fast stock chart capable of drawing very large datasets with high levels of interactivity and have not yet found one. Other than the WriteableBitmap API there is no way to draw quickly in both WPF and Silverlight. The best solutions to achieve real-time visualization have used DirectX (on the desktop). On the web I haven't yet seen a solution using the 3D API but would be interested to give it a go!

    Best regards,

     

    Monday, January 16, 2012 5:18 AM
  • Yes, I checked your chart. Works ok. It needs some more work though but it looks very promissing.

    Thursday, January 19, 2012 2:14 PM