none
Directshow/Webcam in WPF Natively - Test Project

    General discussion

  • I have noticed a lot of post's around the internet of people wanting  better Directshow support within WPF.  Whether its assembling their own graph or using a webcamera, this is pretty much impossible with just WPF.  Microsoft's solution has been to "use interop".  But using interop, you can't use WPF to the fullest (ie, use video on 3D) and the HWND Host likes to sit "on top" of my wpf which makes layout a nightmare.

     

    Some people have used DirectShow .Net opensource libraries and used the SampleGrabber to grab samples, load them into bitmaps in WPF.  Unfortunatly this is slow/cpu taxing as *** because WPF makes you create a whole new bitmap each time (wheres the low level pixel access??).

     

    So here is my work-a-round.  I admit, its a nasty mofo of a hack, but works.  Basically,  I create a WPF MediaElement.  I set its source to "MediaBridge://Something".  When MediaElement looks up in the registry for the protocol handler of "MediaBridge", it loads up my special source filter.  Then my source filter does a callback to the main application, passing reference to the filter graph.  I then modify the filter graph, adding my own source filter (ie, webcam) and arranging my own graph.  WPF then fires up the graph and voila!

     

    I only spent a few hours on it, so be nice when you see the source. 

     

    **IMPORTANT**

    Make sure you register "MediaBridgeSourceFilter.ax" with "regsvr32 MediaBridgeSourceFilter.ax"  and if your are on vista, make sure you register it as an Admin (right click cmd, Run as Adminsitrator).  I left the compiled versions in the zip file incase ya'll dont have the DirectShow SDK or VC++ installed.

     

    Again, this is a total hack so don't blame me if it blows up your computer and kills your mom.  We have to live with such hacks until Microsoft opens up the MIL Sad

     

    Feel free to contact me at: jeremiah.morrill@gmail.com

     

    http://jmorrill.hjtcentral.com/Home/tabid/428/EntryID/15/Default.aspx

     

    PS - The Demo app loads up the first capture device (ie webcam) from your system and puts it in the media element.  Its easy enough to add your own source filters and what not.

    Thursday, May 17, 2007 7:28 AM

All replies

  • I had some issues fighting with what appeared to be a fake overlay effect. Try the following code to get the button be set on the webcam stream:

     

    Code Snippet

    <Canvas>

       <Button Canvas.ZIndex="2" Name="btnStart" Content="Start" Width="60" Height="40" VerticalAlignment="Top"/>

       <MediaElement Canvas.ZIndex="1" Name="BridgedElement" LoadedBehavior="Manual"/>

    </Canvas>

     

    Thanks for this great hack before a better world.

    Saturday, May 19, 2007 9:53 PM
  • Wow, this is exactly what I've been looking for.  If this works, you will truly be my hero.  Seriously.

     

     

    Wednesday, May 23, 2007 10:58 AM
  • It works Smile, though there was one guy that had problems using it with a TV tuner.  I suggest testing the graph first with the EVR renderer filter (as thats the one WPF uses) and see if everything is kosher.

     

    I know its a hack of the worst kind...but ya gotta do what ya gotta do to make ish work, right?

     

    I have more info (and another hack) on my blog if you wanna peep it.

     

    http://jmorrill.hjtcentral.com/

     

    -Jer

    Wednesday, May 23, 2007 11:57 AM
  • Hi

    I just had to say.... that is absolutely brilliant!

    regards

    Steve
    Monday, July 09, 2007 6:54 AM
  • Thanks for the props dude.

     

    Maybe there will be a real fix in .NET 3.5...Hope we don't have to wait for 4.0 =X

     

    -Jer

     

    Monday, July 09, 2007 10:15 AM
  • It's good work, but I have problems with this application. The only string that doesn't work is:

    hr = captureGB.RenderStream(PinCategory.Preview, MediaType.Video, filter, null, renderer);


    Application falls down with "access violation error, memory may be corrupter". If I replace in this string word "renderer" with "null" - it's work, but shows nothing. If I use graphBuilder's function Render to render for file instead of this it's work!

    What's wrong? Maybe this is a result of updates? I have DotNet 3.0 hotfix installed (dotnet v.2.0.50727) as Vista update.

    Other capture applications works fine i.e. Media player calssic, VirtualDub..., web camera is the only capture device in the system.
    Friday, August 24, 2007 7:51 PM
  • I really can't say what the issue is, but at least you got it to work Wink.

     

    That capture code was a snippet pretty much copied and pasted from the DShow.NET samples.  I actually had never used the capture graph builder before this.

     

    -Jer

    Friday, August 24, 2007 9:26 PM
  • Hi

    Has anyone got a code snippet they can post that shows how you use this hack to play a WMV. For some reason when I try and do the following I end up with a DisconnectedContext error. I can get the example graph working with the default video source without any problem.

    int hr = 0;

           
                //Convert pointer of filter graph to an object we can use
                graph = (IFilterGraph)Marshal.GetObjectForIUnknown(GraphInfo.FilterGraph);

                graphBuilder = (IGraphBuilder)graph;


                IBaseFilter renderer;

                //Find WPF renderer.  It's always named the same thing

                hr = graphBuilder.FindFilterByName("Avalon EVR", out renderer);
                //hr = graphBuilder.FindFilterByName("Enhanced Video Renderer", out renderer);

                string filename = @"C:\Documents and Settings\Steve\My Documents\My Videos\therunningman.wmv";
                hr = graphBuilder.RenderFile(filename, filename);
                DsError.ThrowExceptionForHR(hr);

                hr = (graphBuilder as IMediaControl).Run();
                DsError.ThrowExceptionForHR(hr);
    Sunday, September 02, 2007 11:49 PM
  • The stock MediaElement should be able to play WMVs, but if you are trying to inject extra filters in there for other purposes, this hack might be perfect for you (if you can get it to work).

     

    Dunno if this is the problem, but try changing the:

     

    hr = graphBuilder.RenderFile(filename, filename);

    To:

     

    hr = graphBuilder.RenderFile(filename, null);

    Monday, September 03, 2007 12:12 AM
  • Hi

    Yes, I'm wanting to get a custom filter of my own into the graph hence the interest in using the hack. Settng the second parameter to null didn't make any difference I'm afraid.

    One thing I've noticed from the looking at the graph constructed with the videodevice example is that there is an additional Windows Media Video Processing DMO in the graph that does not appear when trying to render an existing AVI or WMV.

    Have you been able to use your hack to render an existing file?


    Monday, September 03, 2007 12:20 AM
  • I didn't do much testing on the code, but I didn't have any issues rendering a couple XViDs off my hard-drive.

     

    Are you just having issues with WMV files? 

     

    I saw that "Windows Media Video Processing DMO" also.  I'm not sure what that's all about.  Try adding the graph to the ROT (DShow.NET has the helper functions I believe) and use graphedit to connect to the "remote graph".  You can then manipulate the graph and play around with what works.

     

    Just a stab in the dark, but MediaElement uses the Windows Media Player, and ever since WMP 7.0 it has had special handling of ASF(WMV) files, unlike v6.0 which was a pure DirectShow player.  Maybe that has something to do with it.

    Monday, September 03, 2007 1:45 AM
  • Hi

    Thanks for that. I tried with DV AVI as well and that results in an error that crashes my app. I'll dig out an xvid and see what happens there. Thanks for your suggestions.

    regards

    Steve
    Monday, September 03, 2007 2:03 AM
  • Hi,

     

    I am having the same "Disconnected Context" problems with very similar code to your's. Did you find a reason for the errors in the end or a fix?

     

    Regards,

     

    Jon Hook

    Friday, September 14, 2007 10:46 AM
  • Didn't noticed this the first time I looked at cheesie's snippet.

     

     hr = (graphBuilder as IMediaControl).Run();

    I dont think you should execute the Run on the IMediaControl.  WMP/WPF handles this already.

     

    -Jer

     

    Friday, September 14, 2007 10:54 AM
  • Thanks alot, all works fine now (and makes sense why!).

     

    Jon Hook

    Friday, September 14, 2007 10:57 AM
  • Great!  Out of curiosity...and if you are not bound by confidentiality, what are you using the MediaBridge for?

     

    One thing I didn't find out until later is WMP seems to create the graph with an STA COM thread, so you won't be able to talk to you DShow filters after it is initialized (because you need the same thread that created the COM objects).  This can be troublesome for some setups, like changing channels on a TV tuner.

     

    If anyone knows a good workaround to the STA thread thing, please lemme know.

     

    -Jer

     

    Friday, September 14, 2007 11:02 AM
  • Hi

    Right, I think I get what's going on then, however I still can't get my graph to show any video. It builds and can be seen in Graphedit but does not run.

    Would you be in a position to post a snipett?

    regards

    Steve
    Monday, September 17, 2007 5:29 AM
  • Hi

    Thanks for that very useful piece of information. So what you're saying is that when your call back is called, we're actually executing in a different thread context compared to the main application?

    regards

    Steve
    Tuesday, September 18, 2007 1:47 AM
  •  

    You're not using IMediaEventEx:: SetNotifyWindow are you?  If you call that, the video won't start.

     

    --nick

    Tuesday, September 18, 2007 2:30 AM
  • Hi Nick

    Thanks for your reply. No I'm not using the interface. I think what I'll do is put a quick example project together and post it with a link in case anyone is interested in looking at it.

    regards

    Steve
    Tuesday, September 18, 2007 3:14 AM
  • I'm writing a VJing application for an interactive table (microsoft surface alike) based on WPF. I'm using MediaBridge to let me display a DirectShow graph containing freeframe effect filters.

     

    Therefore it's pretty bad news that I won't be able to change filters after initialisation! Does your second solution, writing to the bitmap souces's buffer, have similar behavior?

     

    Regards,

     

    Jon Hook

     

    ------------

    The code I have working...

     

    int hr = 0;

    //Convert pointer of filter graph to an object we can use

    graph = (IFilterGraph)Marshal.GetObjectForIUnknown(GraphInfo.FilterGraph);

    graphBuilder = (IGraphBuilder)graph;

     

    //Find WPF renderer. It's always named the same thing

    IBaseFilter renderer;

    hr = graphBuilder.FindFilterByName("Enhanced Video Renderer", out renderer);

     

    // Add FreeFrame effect to graph

    Guid FFguid = new Guid(FF_GUID);

    comtype = Type.GetTypeFromCLSID(FFguid);

    freeFrame = Activator.CreateInstance(comtype);

    hr = graphBuilder.AddFilter((IBaseFilter)freeFrame, "FreeFrame");

    DsError.ThrowExceptionForHR(hr);

     

    // load a file to be rendered by the video player

    string filename = @"C:\q3.avi";

    hr = graphBuilder.RenderFile(filename, null);

    DsError.ThrowExceptionForHR(hr);

    DsROTEntry entry = new DsROTEntry(graph);

     

    // set filter params

    PeteWaveEffectPlugin effect = new PeteWaveEffectPlugin();

    plugIn = freeFrame as IFFPlugMain;

    plugIn.SetPlugin(@"C:\PeteWave.dll");

    plugIn.SetParameter(1, 0.1f);

    Tuesday, September 18, 2007 9:20 AM
  • I'm not sure, as I'm not the resident expert on COM and threading apartments...

     

    But maybe when you init freeFrame COM object, you can init it under an MTA thread.

    Code Snippet

     

    object freeFrame;

    Thread t = new Thread((ThreadStart)delegate

    {

    freeFrame = Activator.CreateInstance(comtype);

    };

     

    t.SetApartmentState(ApartmentState.MTA)
    t.Start();

    t.Join();

     

    //Rest of the graph init

     

     

    Then when you need to talk to the freeFrame object...you might have to do something like:

     

    Code Snippet

     

    Thread t = new Thread((ThreadStart)delegate

    {

    freeFrame.DoSomething();

    };

     

    t.SetApartmentState(ApartmentState.MTA)
    t.Start();

    t.Join();

     

     

     

    Tuesday, September 18, 2007 4:24 PM
  • The bitmapbuffer solution will work for sure, but there is more overhead with it because of the Invalidate() of the <Image/>.  Something WPF does on that operation eats up SOME cpu usage, but is still faster than doing butt-loads of BitmapSource.Create() operations.

     

    -Jer

     

    Tuesday, September 18, 2007 4:33 PM
  • Hi Jon

    Thanks for posting that snippet. It turns out the if I try to play a WMV movie I see nothing on the screen, but playing a DV AVI works absolutely fine.

    Futher investigation reveals that the
    - WMV graph appears to build ok when looking at it in graphedit. There is a source -> WMVVideo Decoder DMO -> Avalon EVR  (note that this video file has no audio). Trying it with sources that do have audio produce the same results.

    - trying to rewind and replay the graph using the Position property and the Play() method of the mediaelement results in the application terminating.

    - Comparing this WMV graph to one with an AVI shows one very interesting difference. The graph with a DV AVI (ie the one that works) has a WMPlayer Video Processing DMO in between the DV Decoder and the EVR.

    So, it looks to me like there is something else required in the chain for WMV's to work. Does anyone know what the GUID is for the WMPlayer Video Processing DMO?

    regards

    Steve
    Wednesday, September 19, 2007 3:38 AM
  • Hi Jeremiah

    I had a look at your suggestion for trying to create a filter under MTA context. I certainly couldn't get it to work. The result was that none of my graph got built.

    As far a a solution goes I'm wondering if there is an option for a very lightweight pass through filter that actutally does nothing other than call back everytime a sample moves through the graph or according to some timing schedule. Most of the time there would be no action in the call back, however if you wanted to it would be possible at that point to make changes to filters because the thread context was correct. Not being at all experienced with threads I'm not sure if this would work though it sounds feasible.

    regards

    Steve

    Wednesday, September 19, 2007 4:46 AM
  • You can always add the SampleGrabber filter to your graph, which will make a callback for each sample in your graph.  You can find examples in the DShow.NET (http://directshownet.sourceforge.net/) samples.
    Wednesday, September 19, 2007 5:31 AM
  • Hi

    Good suggestion. I tried that and unfortunately it didn't work for me. I get a NullReference exception when trying to cast to the interface of the filter I'm wanting to modify. If I used a reference to the filter instead then I end up with a
    System.InvalidCastException

    System.InvalidCastException was unhandled
      Message="Unable to cast COM object of type 'DirectShowLib.SampleGrabber' to interface type 'DirectShowLib.ISampleGrabber'. This operation failed because the QueryInterface call on the COM component for the interface with IID '{6B652FFF-11FE-4FCE-92AD-0266B5D7C78F}' failed due to the following error: No such interface supported (Exception from HRESULT: 0x80004002 (E_NOINTERFACE))."

    I added calls to System.Threading.Thread.CurrentThread.ManagedThreadId in various parts of the application to confirm the context. I found that the UI thread of my app was 10. The BridgeCallback method was 11 and the sample grabbers call back threadId was 16.

    However, the really interesting thing was I tried doing the following as a method called from a button click (ie not in the call back of the sample grabber)
    (graphBuilder as IMediaControl).Pause()
    and the graph paused??

    So I took it a bit further and did this as a method called from a button click
    IBaseFilter dummy;
                int hr = graphBuilder.FindFilterByName("Avalon EVR", out dummy);
                FilterInfo pinfo;
                hr = dummy.QueryFilterInfo(out pinfo);
                Console.WriteLine(pinfo.achName);
    and it returned the corrent name.

    So, now I'm really confused. It's like some filters / interfaces are accessible and others are not?

    Can anyone shed any light on this?

    regards

    Steve
    Wednesday, September 19, 2007 11:55 PM
  • (Apologies for taking the thread somewhat off topic)

     

    Jeremiah,

     

    I've moved over to using the bitmap buffer solution now. I have based some code on your WPFInterop project. The following code attempts to redraw to the bitmap buffer from a sample grabber callback.

     

    The code only seems to work if the pointer to the buffer is regenerated for each frame - hence everything is quite slow!

     

    Do you have any idea what could be wrong, in your code you seem to have separate init and update based functions which seem not to have this problem?

     

    Regards,

     

    Jon Hook

     

    Code Snippet

    unsafe int ISampleGrabberCB.BufferCB(double sampleTime, IntPtr pBuffer, int bufferLen)

    {

    mediaControl.Pause();

     

    this.Dispatcher.Invoke(DispatcherPriority.Normal, (UpdateImageDelegate)

    delegate()

    {

    // when this statement is removed code runs fine..

    if (bitmapSource == null)

    {

    InitBitmapBuffer();

    }

     

    CopyMemory(bitmapSourceBuffer.BufferPointer, pBuffer, (int)bitmapSourceBuffer.BufferSize);

     

    InvalidateVisual();

    });

     

    mediaControl.Run();

     

    return 0;

    }

     

    private void InitBitmapBuffer()

    {

    int width = 512;

    int height = 512;

    int bytesPerPixel = 3;

    int stride = width * bytesPerPixel;

    int totalPixels = width * height * bytesPerPixel;

     

    if (dummyPixels == null || dummyPixels.Length != totalPixels)

    {

    dummyPixels = new byte[totalPixels];

    }

     

    // TODO: Use a bitmap that we can always alter the bits instead of having to recreate it

    bitmapSource = BitmapSource.Create(width,

    height,

    96,

    96,

    PixelFormats.Bgr24,

    null,

    dummyPixels,

    stride);

     

    Source = bitmapSource;

    bitmapSourceBuffer = new WPFInterop.BitmapBuffer(bitmapSource);

    }

     

     

     

    Monday, September 24, 2007 1:43 PM
  • A solution I have found is to use the Stream Buffer Engine and create a completely separate graph as the source.

    Then I just add the source filter to the MediaElement graph and set it to the filename I set in the sink with LockProfile.

     

    I use this method for digital tv graphs mainly. Not sure what sort of performance you could get out of it for other sources.

    Wednesday, January 09, 2008 3:54 PM
  • System.InvalidCastException was unhandled
      Message="Unable to cast COM object of type 'DirectShowLib.SampleGrabber' to interface type 'DirectShowLib.ISampleGrabber'. This operation failed because the QueryInterface call on the COM component for the interface with IID '{6B652FFF-11FE-4FCE-92AD-0266B5D7C78F}' failed due to the following error: No such interface supported (Exception from HRESULT: 0x80004002 (E_NOINTERFACE))."

    Hi Steve,

     

    Did you ever find out why you were getting the above error. I am getting the exact same thing. I am very new to the DirectShow thing. I am using DirectShowLib in managed C++..

     

    Creating a GraphBuilder, Creating a SampleGrabber, configuring it, then adding it to the GraphBuilder. I get the callbacks, but when I try to do:

     

    AMMediaType^ mt = gcnew AMMediaType();

    int hr = this->SampleGrabber->GetConnectedMediaType(mt);

     

    in the BufferCB function, I get the NO_Interface error.

     

    In the native DirectShow, this works fine. Can't get it to work with DirectShowLib though.

     

    Any help would greatly be appriciated.

     

    Thanks,
    Chris

     

     

    Saturday, February 09, 2008 3:42 AM
  • Your error could be caused because you are using the wrong apartment state on your thread.  For testing, try creating everything (ie graph, filters, etc) under an MTA thread and see if you experience the same problem.

     

    I'm saying this under the assumption that you are not using this in WPF.

     

    -Jer

    Saturday, February 09, 2008 5:23 PM
  • Thank you Jeremiah!

     

    That worked and your assumtions are correct. I'm not using WPF. I swear I thought I was finally joining the correct decade by porting our stuff from MFC to .NET WinForms, but it seems I am still way behind the curve by jumping in to WPF.

     

    I can't wait till the day that I can use C# exclusively. My current project is a development/test environment for our embedded C code. The interface is better, but it is still kind of clunky.

     

    Thanks for all of your contributions.

     

    Chris

     

    Saturday, February 09, 2008 6:53 PM
  • I have started v2 of this control to make it easier for developers to use.  Please refer to this link for info:  http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=3079089&SiteID=1

     

    -Jer

    Friday, March 28, 2008 12:41 PM