locked
Question About MediaElement Behind-the-Scenes

    Question

  • Greetings -

    Question for the MS gods (or anyone else who might know), as I'm making a custom control that behaves in many ways like MediaElement and want to confirm the best way to do this.

    Is MediaElement in WinRT essentially a wrapper for a SwapChainPanel whose D3D swap chain is drawn to by the media engine? 

    That's how I'm implementing my custom MediaElement, with good success so far, but before I went further I just wanted to make sure I was on the right track. If there's a more efficient method of making a MediaElement control I'd be very grateful if someone let me know. Thanks!

    Peter




    Thursday, December 18, 2014 2:05 AM

Answers

  • You're on the right track. The only way to get reasonable video without one of the built in media controls is to use DX.
    • Marked as answer by pnm655 Thursday, December 18, 2014 3:32 AM
    Thursday, December 18, 2014 3:24 AM
    Owner

All replies

  • You're on the right track. The only way to get reasonable video without one of the built in media controls is to use DX.
    • Marked as answer by pnm655 Thursday, December 18, 2014 3:32 AM
    Thursday, December 18, 2014 3:24 AM
    Owner
  • Thanks for the feedback (even though you dodged my question ever so slightly ;))  It helps a lot to know this is the right approach. What I'm actually doing is mirroring playing video on two screens, so it's extremely useful to be able to share the same swap chain amongst two XAML elements. In fact, it seems to work flawlessly even though the second window is on its own UI thread. Of course, my assumption is that the two screens need to be on the same D3D device and thus the same video display adapter, so I would suspect this would not work for, say, Miracast projection. I don't have a device to test that theory with yet but does that sound likely?

    Also, if I could expand the scope ever so slightly to desktop apps, would you say my best option on the WPF desktop side is D3DImage? My construct is working ok on the desktop side using D3DImage (obviously not using media engine but rather media foundation with a custom VFR), but not quite as well as on the RT side using SwapChainPanel. I assume that's because on the desktop side I need to keep alerting the D3DImage to redraw itself for each frame on the UI thread, whereas on the WiNRT side using SwapChainPanel, all I need to do is update the swap chain and the UI thread is blissfully ignorant of what's going on.

    On the desktop side, would it be better perhaps if I had D3DImage on an independent timer that simply started and stopped when the media was playing or not playing, respectively? I suppose avoiding all those switches back and forth between threads could make a big difference. On the other hand,  I would think vsync artifacts could get quite bad without perfect timing, given that with D3DImage I'm not using the swapchain but just a D3Dsurface.

    Or is there a better way still?

    (Incidentally, I did consult the .NET reference source for WPF MediaElement but there seems to be underlying functionality that's either not public or I couldn't find it).

    Thanks again.

    Thursday, December 18, 2014 9:25 PM
  • What did I dodge? You'll need to explain more clearly. If you're trying to write your own media element to host in Xaml you'll need to use DX interop and one of the Xaml SwapChain classes. This is essentially what the MediaElement does (except it has internal access to DX and doesn't go through SwapChainPanel).

    I would expect you'll get the best results by using separate controls with different swap chains for the different windows. I suspect you're getting software rendering in the second screen right now rather than hardware rendering in both.

    For help with desktop apps please ask in the desktop (in this case WPF) forums.

    --Rob

    Friday, December 19, 2014 12:48 AM
    Owner
  • Hey Rob, sorry I didn't mean to sound critical. I was just joking that maybe you didn't want to say what's going on inside of the RT MediaElement because it was not public knowledge. But thanks for confirming that it's operating in a similar fashion to what I'm doing.

    Interesting what you're saying about using two swap chains for the two screens. Actually I'm getting VERY good results right now doing it with only one. Also I would think that the overhead from having to render the video frame to two different swap chains would be significant. But really I'm quite satisfied with the RT performance right now. Stunned actually. I don't know any other platform where I can manipulate playing high definition video in real time mirrored on two screens while still having access to the whole UI as well. 

    It's the WPF desktop version of my app that is not giving me the best performance and I thought by trying to compare and contrast the two approaches I could figure out why. I'll see if the folks in the WPF forums can help though. Hopefully they won't send me back here. :) 


    Friday, December 19, 2014 1:56 AM
  • To quickly follow up, turns out you were right about using two swap chains. Having the two threads sharing the same swap chain was definitely leading to problems. Not sure if it was hardware vs. software rendering or what exactly, but switching to two swap chains and two concurrent calls to TransferVideoFrame did the trick and I also did not notice any performance hit. So, that seems to be the answer. Thanks again.
    Tuesday, December 30, 2014 3:33 PM