Saturday, August 18, 2012 1:26 PM
I have a problem with the SwapChainBackgroundPanel, I'm trying to create a big panorama of mixed Direct2D and Xaml elements. I tried to do this with a VirtualSourfaceImageSource but the constant flickering and lag made that solution unacceptable. So I created a Window with a SwapChainbackgroundPanel and put a Grid inside it and loaded all xaml elements into that grid and positioned all of them using Margin values. After that I implemented a scrolling effect by moving the Xaml elements with a scrollviewer and the direct2D elements under them with simple matrix transformations calculated from scrollviwer position. I also created another version with only a scrollbar and translatetransform for xaml. I tried rendering the Direct2D content in every CompositionTarget.Rendering event but the movement of the xaml elements are moving much more "slowly" than the Direct2D elements and this "sliding" makes this solution as unacceptable like the VSIS version was. (Note that I tried to call render for directx also before/after I set the translation for Xaml elements.) Did anyone experienced anything like this? I'd appreciate any advice on this topic. (I'm on RP now but it's been tested on RTM with the same results)
Here is an example video to illustrate the problem (the yellow rectangle is a XAML rect, the background is direct2d): (edit, it was a wrong video link) http://www.youtube.com/watch?v=TCI8nUtNn00
- Edited by David Serfozo Saturday, August 18, 2012 6:10 PM
Tuesday, August 21, 2012 12:51 AMModerator
Thanks for posting the video that was very helpful in visualizing the issue that you are seeing. Unfortunately I think that you are really pushing the technology beyond the original intent. I don't know that we ever considered the possibility of developers wanting to scroll a D2D background with a XAML foreground. Maybe someone in the community can help you come up with a really creative solution but unfortunately I just can't think of any way that you could possibly sync the two in a meaningful way that would work across all devices.
Maybe if you could hook the XAML swap chain and somehow figure out when the XAML present is going to be called you could synchronize your D2D calls to it. However, I don't think that this is possible. Keep in mind that when mixing XAML and DX technologies the DX stuff gets drawn first and the XAML is drawn on top.
I wish I had something more for you.
Windows Media SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/
- Proposed As Answer by James Dailey - MSFTMicrosoft Employee, Moderator Tuesday, August 21, 2012 12:51 AM
Tuesday, August 21, 2012 7:08 AM
How is your DirectX rendering synchronized? Is appears your DirectX is running "on it's own", and not being called to update from the "composition target". (What I mean is that you can set up to receive an event when the Xaml is rendering, where you can do yours too, which will keep things in sync)
Two ways to tell:
1. Is your "update/render/present" in a free-running loop (some examples do this).
2. Look where your swapchain is being set up... most likely in "CreateWindowSizeDependentResources" (in the base object) assuming your following any of the samples that actually have a SwapChainBackgroundBuffer.
If you find "CreateSwapChainForComposition" in your code you are (looking at your video, I don't believe you are)
If you find "CreateSwapChainForCoreWindow", (as in the "stock" 3D template), you are not.
The sample "XAML DirectX 3D shooting game" shows how to set it up. Just search for "composition", and it will show just about everything you can't find in the functions listed above.
Oh... if you are not synced, you could also be drawing way too much on the DX side, so if this indeed your problem, this fix should make your day better.
Hope this helps
- Edited by James Tomasko Tuesday, August 21, 2012 7:13 AM more info
Tuesday, August 21, 2012 3:47 PM
thanks for the comments. Actually for the POC which produced the above video, I used one of the SharpDx samples as a start and I'm creating the swapchain using the CreateSwapChainForComposition method. The render is called in the CompositionTarget.Rendering event according to the MSDN documentation, available here: http://msdn.microsoft.com/en-us/library/windows/apps/hh825871.aspx, stating: "You update the SwapChainBackgroundPanel based on the needs of your app, and not the updates of the XAML framework. If you need to synchronize the updates of SwapChainBackgroundPanel to those of the XAML framework, register for theWindows::UI::Xaml::Media::CompositionTarget::Rendering event. " (but I tried it several other places also without luck). So I really don't get it why the whole thing is out of sync, I don't even draw more than some rectangles with direct2d on a mid level GPU and CPU.
Btw. it makes me really disappointed that these DirectX+XAML tools are limited to a few scenarios. I'm trying to create a high performance scrolling experience but all the available solutions (SIS, VSIS, SwapChain...) were performing really bad under these conditions. I'm just wondering how all these stuff will work on an much more power limited ARM device...Of course it's more than possible I'm using some of the APIs the wrong way, but I only have the MSDN docs and the samples to start with and based on those I'm doing everything "by the book".
Tuesday, August 21, 2012 5:10 PM
Would it be possible for you to share your POC apps (e.g. post on SkyDrive)? The VirtualSurfaceImageSource one would be interesting as well if you could share both - we've seen a number of apps successfully use VSIS for high-performance scrolling of many pages of content on low-power devices, so there may be some suggestions we could make.
Wednesday, August 22, 2012 10:57 PM
I have just started using SIS and have not experienced any flickering. I haven't tested for lag between xaml and d3d renderings yet though.
I believe the SwapChainBackgroundPanel uses DirectComposition under the hood and the CompositionTarget::Rendering event is only a UI thread sync IIRC.
There is latency in video rendering. If you have to devices rendering independently, it is possible for one to be one or two frames behind (as gpus work asynchronously with the CPU), which could be up to 32ms behind if you are targeting 60fps. This is even more probably given you _are_ using a "SwapChain" ;)
Given SIS and VSIS are more tightly controlled by XAML, I do expect better results there for syncing with XAML content. But as I said before, I never tested this scenario myself. The bing maps application is a SIS or VSIS and was told (read: hearsay) the pushpins are xaml.
Thursday, August 23, 2012 7:38 AM
I've made both POC apps available on my skydrive: https://skydrive.live.com/redir?resid=A02346357B795002!1337&authkey=!AGMv4BvztK_oJAU . I've tried both on a Samsung BUILD tablet, however the difference between a lower and higher performance device only visible in the VSIS version (the SwapChain is wrong on both).
I'd be really happy if you could say something about them.
Thank you very much,
Thursday, August 23, 2012 7:42 AM
Sorry for the misunderstanding...with the SIS I've no flickering problems it's just eating up too much memory and takes to much time to render it...Because I'm displaying a huge canvas it seemed to be a logical choice to use the VSIS but that's the one causing flickering. The update rects are clearly visible on a lower powered device when I'm scrolling fast and it's ruining the UX :(. Btw. I was expecting something like shown on the attached vid when I tried the SwapCahinBackgroundPanel but unfortunately I hadn't any other ideas. And indeed when using the SIS or VSIS the sync is perfect but both has some minor disadvantage in my case :(.