Need help adding overlay graphics to live video rendered via Microsoft Media Foundation / Direct3D RRS feed

  • Question

  • Hello,

    I have an app running on Windows 7 which is using a WPF window to host live video coming from a webcam.  Everything is working well, but now I need to add overlay graphics / text on top of the live video.  I've tried rendering the graphics directly via WPF with no luck, presumably due to "airspace" issues -- rendering WPF graphics on top of non-WPF rendered content. 

    I suspect I must do the overlay within MMF / D3D.  My problem is I know very little about these technologies.  I learned just enough to modify one of the existing MMF sample programs to perform the video rendering.

    The rendering code has a class derived from IMFSourceReaderCallback.  As frames come in from the source, OnReadSample() renders each frame with a helper class.

    The helper class uses an object which implements IDirect3DDevice9 and invokes several methods to render each frame as follows (some function parameters omitted for brevity):

    IDirect3DDevice9 * m_pDevice;



    m_pDevice->StretchRect(Surface, BackBuffer);  // Surface is the back buffer of an "additional swap chain" created on the device


    I'd like to modify this code to essentially do another blit of overlay graphics / text, but I don't speak enough D3D/MMF-ese to know how to do this.

    Ideally I'd like to generate the overlay graphics with WPF style drawing code rather than D3D-style code if possible.  I'd like an interface where the client (overlay supplier) can supply a new overlay via WPF as desired, and that overlay will be blitted over all incoming frames as necessary until a different overlay is supplied.

    A longer term requirement is for the live video window to support digital zooming.  I don't think the webcam directly supports areas of interest within the frame, so I suppose this needs to be done via cropping out an area of interest in software and then stretch blitting to fill the window.  I'd like to put in some support for this if possible.

    Can anyone point me to some good tutorials or example code that can help get me up to speed quickly on implementing these things?  BTW, I'm not married to MMF / D3D; if there's an easier way to do this, I'm open to other suggestions.

    Thank you very much for any guidance you can provide!
    Wednesday, March 9, 2011 4:08 AM

All replies