Media Extensions; which path to take?


  • Hello all,

    I am trying to write a directx windows store app. This app will use most of the screen to display directx graphics. A small portion of the screen would be dedicated to streaming video. The video and the graphics will be displayed simultaneously. I want to capture the video and merge it with my directx graphics and send it across the network from one peer to another.

    I went though the media capture sample and the media extensions samples available online.. If I go down the path of media transform, I have access to the raw video buffers in the media transform DLL but how do I gain access to them in my main app window. Most of the "MediaCapture::StartRecordToCustomSinkAsync" style APIs expect a media profile. I do not want to encode the video yet. I want to only grab the raw frame and merge it with my directx graphics and then encode and send it.

    Summarizing, I need access to the raw video frames in my app not just in the media transform DLL. How do I do that?

    Thank you all in advance.

    Monday, November 12, 2012 7:10 PM

All replies

  • Hello,

    The MediaCapture element is a bit limited when it comes to raw data. We do have general guidance on this scenario. You should create your own scheme handler and specify the media type. This can be raw data.

    That said I would recommend that you consider using the HW accelerated encoding and decoding of h.254. This is automatically done for you if you use the standard features of the MediaCapture source. We added "real time" h.264 codec capabilities that allow for live communication scenarios such as the one you describe.

    Please take a look at this sample. I think it will give you a good place to start:

    Real-time communication sample


    I hope this helps,


    Windows Media SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/

    Tuesday, November 13, 2012 12:26 AM
  • Thank you for your reply.

    I did go through the scheme handler scenario in the media extension sample as well as in the simple comm.

    In the latter case it still uses a call to "StartRecordToCustomSinkAsync" with the media encoding profile. I do not want to encode my video, decode it then merge it with my directx graphics and then encode it again to send. I want to be able to get the raw video frame, merge it with my directx graphics and then call StartRecordToCustomSinkAsync to encode it.

    In the media extensions sample the geometric source generates frames that do not contain video that has been captured from the web / inbuilt camera. If you could give me some guidance on grabbing the video camera's output and send it to my main program via a scheme handler I would appreciate it. 

    In the main program I would try to then merge it with my directx graphics end then encode it.
    Tuesday, November 13, 2012 1:51 AM