locked
Capturing Webcam Video Frames - Windows 8 Store App

    Question

  • We are building an Augmented Reality application and require the video frames from a webcam source. We have a native only project that has been used elsewhere and do not want to convert this to XAML/C# interop if possible. This question is for native only.
    Our understanding is that the best way to grab the frames is to use a custom MFT to pipe the frames to where we need them. In our case this is a DX texture. Are these the correct steps?
    1. Create a MediaCapture object, and call InitializeAsync with settings for Video only and the camera device id.
    2. Call StartPreviewAsync().
    3. Finally, add call AddEffectAsync with a custom MFT object.
    Calling StartPreviewAsync() fails when the MediaCapture object is not set as a source in a XAML CaptureElement. We'd like to start a preview and capture the frames in the MFT without actually having the XAML output source or any official output source. Is StartPreviewAsync the correct way to do this? Does it even work this way? We don't want to use StartRecord as we don't want to save the stream anywhere unless we can extract the frames and discard.
    I have looked through the samples/forums and cannot find a concrete example of how to capture raw frames or output to anything but a XAML CaptureElement.
    Firstly, is this the correct approach? If so, why doesn't StartPreviewAsync() work?
    If the process is incorrect, what is the correct process and is there a native example we can reference or be provided with? Is there another technique?
    Wednesday, August 21, 2013 11:19 AM

Answers

  • Take a look at the Media Extensions sample at http://code.msdn.microsoft.com/windowsapps/Media-extensions-sample-8e1b8275 for information on creating your MFT.

    It's not clear if your objection to Xaml/C# is the Xaml or the C#. Xaml is a native technology and can be used You can use in C++ apps without interop to managed code.

    You can use StartPreviewToCustomSyncAsync to preview to your own sink instead of to a Xaml or Html control, or you can record rather than preview.

    --Rob

    Wednesday, August 21, 2013 2:23 PM
    Owner

All replies

  • Take a look at the Media Extensions sample at http://code.msdn.microsoft.com/windowsapps/Media-extensions-sample-8e1b8275 for information on creating your MFT.

    It's not clear if your objection to Xaml/C# is the Xaml or the C#. Xaml is a native technology and can be used You can use in C++ apps without interop to managed code.

    You can use StartPreviewToCustomSyncAsync to preview to your own sink instead of to a Xaml or Html control, or you can record rather than preview.

    --Rob

    Wednesday, August 21, 2013 2:23 PM
    Owner
  • I have no objection to XAML or C#, I have an objection with rewriting the front end of an application. When we first came to the Windows 8 store platform, we chose the native app template and don't want to rewrite this front end to use any XAML/C# or Direct2D surfaces. It has been a long time since I looked at the Direct3D with XAML template, but so far we have been fine without it. From what I understand we can use Media Capture in C++ without XAML.

    I saw a custom Sink written in the Real-Time Communications sample here:

    http://code.msdn.microsoft.com/windowsapps/Simple-Communication-Sample-eac73290

    I don't have much experience with COM objects personally and do not understand why it has to be so complicated to get webcam image data out. That said, I am happy to learn and move forward but I cannot find any relevant documentation that explains how to create a simple media sink. Is is possible to create a simple sink. Is there some documentation that demonstrates how to do this?

    Help is greatly appreciated!

    Edit - I was looking at another example, I am confident I can modify the Grayscale MFT to use for our own needs. The main issue is how to pipe through webcam image data without sending it to an output source. We don't want to record (unless we can save to memory and discard) but again can't find any good examples or how to do any of this without XAML.



    • Edited by MrReapz Thursday, August 22, 2013 4:41 AM
    Thursday, August 22, 2013 4:35 AM
  • Hi MrR.

    Sorry for the confusion. You were very specific about wanting to stay native, and it wasn't clear that you realized that Xaml was native code.

    As I mentioned, it is easier to preview to Xaml but it isn't required. You can call StartPreviewToCustomSyncAsync to preview to your own sync.

    --Rob

    Thursday, August 22, 2013 5:32 AM
    Owner
  • Hi Rob,

    Can Microsoft provide any detailed examples of how to implement a simple custom media sink to use with StartPreviewToCustomSyncAsync? The documentation on the subject is very light.


    • Edited by MrReapz Thursday, August 22, 2013 6:48 AM
    Thursday, August 22, 2013 6:48 AM
  • There is sample code in the Media Extensions sample that should get you almost all of the way there.

    --Rob

    Thursday, August 22, 2013 7:50 PM
    Owner