locked
How to use Media Capture AddEffectAsync with a Media Foundation Transform that uses DirectX / Direct3D

    Question

  • I am trying to implement a store app that displays the device camera preview with real-time effects (filters) applied to it.

    I am porting an existing app that uses OpenGL/GLSL for stream manipulation, so a DirectX/HLSL based Media Foundation  made sense.

    To test this i took the Media-Capture-Sample ( http://code.msdn.microsoft.com/windowsapps/Media-Capture-Sample-adf87622 ), which already uses the Grayscale MFT, and added to the project the Invert directx based MFT taken from Media-Extensions-Sample ( http://code.msdn.microsoft.com/windowsapps/media-extensions-sample-7b466096 )

    I changed the code in the capture sample's AdvancedCapture.xaml.cpp from

    m_mediaCaptureMgr->AddEffectAsync(Windows::Media::Capture::MediaStreamType::VideoPreview, "GrayscaleTransform.GrayscaleEffect", nullptr);

    to

    m_mediaCaptureMgr->AddEffectAsync(Windows::Media::Capture::MediaStreamType::VideoPreview, "InvertTransform.InvertEffect", nullptr);

    The modified sample builds and runs normally but when I try to apply the invert effect, the preview freezes and I get a generic error saying "Fatal ErrorThe Parameter is incorrect". I tried debugging this but could not figure out where this error originated from. Since both Grayscale and Invert are MFT and implement the same interface why would one work and the other not ?


    • Edited by Apollyon G10 Thursday, February 13, 2014 12:22 PM links
    Thursday, February 13, 2014 12:20 PM

All replies

  • I'll ask our media guys to check this out.

    Matt Small - Microsoft Escalation Engineer - Forum Moderator
    If my reply answers your question, please mark this post as answered.

    NOTE: If I ask for code, please provide something that I can drop directly into a project and run (including XAML), or an actual application project. I'm trying to help a lot of people, so I don't have time to figure out weird snippets with undefined objects and unknown namespaces.


    Friday, February 14, 2014 2:02 PM
    Moderator
  • Hello,

    I haven't looked at the "invert" sample but I know that the greyscale sample is in place. When dealing with real time processing typically you don't want to cache any of the frames. I'm just guessing but this might be the problem.

    -James


    Windows SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/

    Friday, February 14, 2014 10:08 PM
    Moderator
  • Thanks for the quick replies.

    I will look into the caching thing. The invert MFT is taken from a sample that works with video playback so it might be locking the frames. The problem is that this is my first foray in windows dev so I am not exactly sure what to look for.

    Also attaching the code of the modified media capture sample:

    https://docs.google.com/uc?id=0B-V5YwH3bIXdN1NDUUcySVRoTDQ&export=download

    Saturday, February 15, 2014 8:55 AM
  • Comparing the two MFTs (grayscale vs invert)

    Grayscale's OnCheckMediaType() checks for NV12/YUV2/UYVY formats,
    while Invert's OnCheckMediaType() returns 'MF_E_INVALIDMEDIATYPE' if the format is anything other than 'ARGB32'

    I am guessing the device camera does not return ARGB format frames, right?
    Are there any existing DirectX samples that deal with YUV textures ?

    Monday, February 17, 2014 8:17 AM