none
How to get images/frames from recorded video with MediaCapture?

    Question

  • I'm trying to make a Augmented Reality kind of Windows 8 Metro app. But i'm kinda having problems figuring out what is the best way to capture images/frames from a video captured with the MediaCapture class.

    At the moment I use MediaCapture to record a video to a stream and use a dispatchtimer to take photo's every 300 milliseconds (it was less before, but since the Consumer Preview it won't work anymore if less than 300ms). The only use of the video recording is to keep the webcam turned on, or else it will continuously turn the webcam on/off

    here is the sample of the code I have made:

    MediaCapture mediaCapture = new MediaCapture();
                    await mediaCapture.InitializeAsync(
                        new MediaCaptureInitializationSettings
                        {
                            StreamingCaptureMode = StreamingCaptureMode.Video
                        });
    
    IRandomAccessStream picturestream = new InMemoryRandomAccessStream();
    IRandomAccessStream videostream = new InMemoryRandomAccessStream();
    await mediaCapture.StartRecordToStreamAsync(MediaEncodingProfile.CreateMp4(VideoEncodingQuality.Vga), videostream);
    
    ImageEncodingProperties iep = new ImageEncodingProperties();
    iep.Height = uint.Parse("600");
    iep.Width = uint.Parse("800");
    iep.Subtype = "JPEG";
    
    DispatcherTimer dispatchTimer = new DispatcherTimer();
                    dispatchTimer.Interval = new TimeSpan(0, 0, 0, 0, 300);
                    dispatchTimer.Tick += async (s, e) =>
                    {
                        await mediaCapture.CapturePhotoToStreamAsync(iep, picturestream);
    
                        //process image
    
                        BitmapImage bi = new BitmapImage();
                        bi.SetSource(picturestream);
                        picture.Source = bi;
                    };
                    dispatchTimer.Start();
    Can anyone tell me if there is a better way to do this? is it possible to get the images/frames from the videostream?

    • Edited by MikeKoeman Tuesday, March 06, 2012 3:21 PM
    Tuesday, March 06, 2012 12:23 PM

Answers

All replies

  • Hello Mike,

    At this time we do not allow access to the underlying media stream from managed languages. If you would like to get access to the underlying video stream you should write your own MFT plug in for the underlying media engine. This must be written in unmanaged C++ using the Metro COM framework. This component can then do all of the AR processing in real time and pass the results back to your managed UI application. We demoed a similar system at the "build" conference. Check out the links below for more info.

    Media capture using webcam sample" found here: http://code.msdn.microsoft.com/windowsapps/Media-Capture-Sample-adf87622

    The build conference "recognizer presentation": http://channel9.msdn.com/events/BUILD/BUILD2011/PLAT-776T

    I hope this helps,

    James


    Windows Media SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/

    Wednesday, March 07, 2012 1:18 AM
    Moderator
  • Will you simply not allow it, or is it possible but not recommended? I've been trying to read MediaCapture.StartRecordToStreamAsync as a stream and convert it into a byte array. I end up with a big byte array containing nothing but zeroes. Is this because it's not allowed, or does it sound like and error in my code. If error in code, will it be able to bit manipulate life video from camera?
    Saturday, December 29, 2012 8:22 PM
  • Will you simply not allow it, or is it possible but not recommended? I've been trying to read MediaCapture.StartRecordToStreamAsync as a stream and convert it into a byte array. I end up with a big byte array containing nothing but zeroes. Is this because it's not allowed, or does it sound like and error in my code. If error in code, will it be able to bit manipulate life video from camera?
    Nice memory leak of a bug if it's a redundant feature that just doesn't work.  I would be interested to know if that returns 0's for that reason too.  Have you found an answer as yet?

    Somewhere out there...

    Tuesday, April 15, 2014 10:40 AM