Capturing Video using WinRT APIs on Windows 8

Question

I am writing native code in C++ to capture video on Windows 8 using WinRT API's.

1. For video capturing I am trying to use IMFMediaSource ( OnReadSample callback to get the data)

   I have referred MFCaptureD3D and MFCaptureToFile sample which works on Windows 7.

   To enumerate the video capture device
   MFEnumDeviceSources is used in earlier versions of the windows.
   What is equvalent API in case of Windows 8 ?

2. Meanwhile I came across MediaCapture class.
   In my case I need YUV data. Can I capture raw YUV data using MediaCapture class or
   It is only to capture encoded data?

   Please suggest which one I should use to capture raw YUV data from WebCam?   
   Thanks in advance.. Looking forward for the help

Answer 1

You can refer to the sample: Media capture using capture device sample and get the answer.
1: function MediaCapture::AdvancedCapture::EnumerateWebcamsAsync() in AdvancedCapture.xaml.cpp
2: As far as know, custom MFT is the only way to get YUV data. GrayscaleTransform in this sample is a good start.

Answer 2

Thanks for the reply.

To understand correctly, you mean to say, we have to capture encoded data from the device and then convert it to desired YUV data format by custom MFT functions. I am worried about performance getting affected here.

Is there any way to get raw YUV frames itself ??

I can use IMFSourceReader to get raw frames but got stuck while enumerating the device (not able to find the equivalent function for MFEnumDeviceSources ).. Pls help...

Answer 3

Using custom MFT(Media Foundation transform) can get raw YUV frames. GrayscaleTransform is a custom MFT sample and can be found in "Media capture using capture device sample"
MFEnumDeviceSources is no longer be supported in metro app, so you should use MediaCapture.
If you want to pass some parameters to the MFT or get some information from it, you can refer to:
http://social.msdn.microsoft.com/Forums/en-US/winappswithhtml5/thread/7f039949-04a8-4457-a8e8-cdbe26e926a5
http://social.msdn.microsoft.com/Forums/en-US/winappswithnativecode/thread/39365c13-3468-4130-a8ed-6bad4df231b1

Answer 4

Thanks for the help. I will go through the links..

Answer 5

Hi..

Using custom MFT ( IMFTransform interface - setInputType,setOutputType,ProcessInput,ProcessOutput API's to get YUV frames from encoded frames ) is a redundant work for my project. Since I need to encode these YUV frames again with our optimized encoders before transmitting to network. Getting captured frames as encoded and decoding it to raw frames is extra work which can affect the overall performance. I am really interested to know why raw YUV frames capture is not supported in WinRT ? Please help me to understand.. 

Answer 6

Generally for capturing app, in ProcessInput function, you can get YUV data. It determined the format supported by your camera and what format you choose.

Answer 7

Hello,

I know it's been a while since you posted that, but I also need raw frame (YUV) in Windows Phone 8.1 (RunTime, Universal App). How did you finally make it ?

Did you use MFT driver, which can affect the performance especially on a Smartphone ? Or did you find another way ?

Since Windows phone 8.1, many API are pretty much the same between Windows 8.1 and Windows Phone 8.1, so I guess you find it on Windows, it will be the same on the smartphone.

Thanks