none
Webcam Preview / Streaming

    Question

  • Hi all.
    Been looking around with the aim of writing an app that can both preview and stream webcam video, but appear to have fallen at the first hurdle.
    All the information I can find seems to be older, using either native calls or DirectShow to perform this, but I've been told that the framework now supports using a WebCam.

    Anyone got any information on how to use a webcam preview, and maybe how to go about streaming it over a network?

    Thanks!
    Sunday, September 24, 2006 12:36 PM

Answers

  • Sorry, I'm afraid you've been misinformed. The framework does not support using a Webcam. This is something we hope to support in a future version, but it will not be part of .NET Framework 3.0.

    Tom

    Monday, September 25, 2006 6:11 PM
    Moderator
  • So I finally got time this weekend to sit down and play with the webcam preview (after getting tired of hearing KC nagging me about doing it) and lo and behold, I got it working.

    I must admit that the Directshow.Net library helped tremendously! Especially their wrapper around the SampleGrabber filter.

    SampleGrabber provides a callback that basically gives you access to the media sample as it goes through the graph. I hooked the SampleGrabber up to a video output pin and registered my callback.

    The callback uses BitmapSource.Create() to create a new bitmap from the current sample and assign it to a BitmapSource dependency property that is bound to an Image on a panel.

    Everytime a new sample comes in, it gets put on the panel. It appears to run as fast as a standard windows webcam preview app...but I can always get some metrics to check Frames Per second and stuff like that.

    Because all WPF sees is an image, I get to do cool stuff like transparency effects and layering, scaling, transforms...all of that good stuff. Of course it won't work in a sandboxed environment because of the unmanaged calls. But it will allow you to do something like put a channel from your tv tuner on the sidebar.

    Like I said before...fun little weekend project

     

    Monday, October 09, 2006 4:28 PM
    Moderator
  • Yes you have to use the dispatcher to make a UI change from a different thread.

     unsafe int ISampleGrabberCB.BufferCB(double SampleTime, IntPtr pBuffer, int BufferLen)
            {
                byte* m_buffer = (byte*)pBuffer;
                img.Dispatcher.BeginInvoke(DispatcherPriority.Render,
                delegate
                {

                   img.Source = null;
                   img.Source = BitmapSource.Create(
                     m_videoWidth,
                     m_videoHeight,
                     96,
                     96,
                     PixelFormats.Bgr24,
                     null,
                     (IntPtr)m_buffer,
                     m_ImageSize,
                     m_stride);
                })
                return 0;
            }

    We got the Dispatcher from the Image control and called it's BeginInvoke method (this allows the function to return without waiting for the dispatcher to finish), and pass it an anonymous delegate (more performant than declaring a delegate and passing parameters to it) that does the work.

    Wednesday, October 25, 2006 2:41 PM
    Moderator

All replies

  • Sorry, I'm afraid you've been misinformed. The framework does not support using a Webcam. This is something we hope to support in a future version, but it will not be part of .NET Framework 3.0.

    Tom

    Monday, September 25, 2006 6:11 PM
    Moderator
  • I'm also looking for something like this, I want to:

    1. Encode a movie/audio-file (from any kind of source) on a server

    2. Stream the media to a client (with WCF?) and display it in a MediaElement.

     

    Step 1 may be skipped, i just want to learn the basics so i can continiue and build a more advanced system.

    Monday, September 25, 2006 6:39 PM
  • MediaElement supports URLs that point to streams. Just put the URL that references the stream on the server in MediaElement and it will work.

    (and if you can host your webcam video as live streaming content on a server, then, yes, you can play webcam video)

    Monday, September 25, 2006 9:36 PM
    Moderator
  • i'm looking to do something like this too, but i'm not going to take the streaming URL approach. instead, i'll have a web cam tethered to my notebook, and it needs to do video processing too.

    so i think i'll have to create a WinForm control that wraps DirectShow for rendering the video, as well as getting access to the lower-level bits for video processing. then i'll have to host that WinForm control within WPF.

    hoping that Media Foundation provides better bits for accomplishing this in the future.

    Thanks, casey

    Monday, September 25, 2006 9:56 PM
  • You can do this, but be aware that you won't have access to the compositing capabilities of WPF for the video.

    This isn't really a question of Media Foundation support, but of WPF support.

    Monday, September 25, 2006 11:53 PM
    Moderator
  • Why not create a WPF control that does what you want? I'm pretty sure from my experience with DirectShow/WMF (Windows Media Format) SDK that it provides a PreviewSink that lets you display a preview of a video stream before and after encoding. Although the PreviewSink was designed to be displayed on an HWND, there's nothing to stop you from giving it an in memory HWND and getting the individual frames as they are passed to the HWND, allowing you to display them on your WPF control.

    Basically, you build a CaptureGraph using a VideoCapture Filter as the Input and connecting it to WMASFWriter Filter for the output. This basically allows you to stream the video over the Net.

    Again, I'm a bit rusty on this but I believe you can get the individual frames in from a stream (via a sink) and use them to create your preview window.

    Tuesday, September 26, 2006 1:14 AM
    Moderator
  • So doing a little more diving into the DirectShow API, I believe it is possible to grab a preview stream and get the frames without relying on an HWND to display them. I've got some old code that wraps directshow (needs to be translated to C#) that I think can form the foundation for a WPF-based Webcam control/application. What say you Mr. Chesnut, up for the challenge?
    Tuesday, September 26, 2006 1:37 AM
    Moderator
  •  Tom Mulcahy - MSFT wrote:

    MediaElement supports URLs that point to streams. Just put the URL that references the stream on the server in MediaElement and it will work.

    (and if you can host your webcam video as live streaming content on a server, then, yes, you can play webcam video)

    I guess this is the wrong forum for this but how do you create a server that can send a mediastream?

    Tuesday, September 26, 2006 1:06 PM
  • DirectShow has already been wrapped here : http://sourceforge.net/projects/directshownet/ ... that is where i would start

    Tuesday, September 26, 2006 2:21 PM
  • to create a stream, you should use Windows MediaEncoder.

    you'll be able to choose the web cam as a source and a url as a target.

    plus it has an SDK, so you could automate part of it if necessary.

    Tuesday, September 26, 2006 2:23 PM
  • Or if you want to program your own Windows Media Encoder for Internet streaming, you would have to create a DirectShow Filter that wraps the Windows Media Format SDK and opens a port on your machine for streaming.

    I just checked, and the AsfWriter Filter included with DirectShow only writes to a file not to an Internet stream as I thought it did before.

    Tuesday, September 26, 2006 3:26 PM
    Moderator
  • Further detail. The CBaseRenderer class helps in creating a streaming WMF Filter (and you can use the InputPin that it provides to retrieve the individual frames as they are passed to the Filter). The frame samples are pushed to the filter by the graph through the InputPin, and the filter is responsible for pushing them to the WMF API (and could possibly provide a callback or event that allows a Winforms or WPF client to receive the individual frames as it is received by the filter.

    Hmmmm...sounds like a fun weekend project...(along with the 50 other ones on my list).

     

    Tuesday, September 26, 2006 4:15 PM
    Moderator
  • I think implementing this as you describe would be very complex. A fun weekend project sounds optimistic.

    Even if you did, you would have several limitations:

    1. Lack of videocard acceleration (alternatively you could do the video processing on the card and then transfer the frames into system memory but this would also be slow)

    2. Since you would be dependent on unmanaged APIs, you couldn't run in the SEE.

    3. Limited control over which frames are used for each composition pass.

    Tuesday, September 26, 2006 7:03 PM
    Moderator
  • Is there not any managed library that has the same functionality as Windows Media Encoder? Working with Com isn't fun after 1 year with .NET
    Tuesday, September 26, 2006 8:35 PM
  • you should be able to just add a reference to the Windows Media Encoder SDKs COM dll, and VS.NET will create an Runtime Callable Wrapper ... which'll let you make calls against the WME SDK as if it were all .NET.
    Tuesday, September 26, 2006 8:48 PM
  • Tom, then what do you suggest we do for lower-level video handling in WPF? ... other than wait.

    Tuesday, September 26, 2006 8:52 PM
  • WPF is not designed to do lower-level video handling. I'd recommend you either

    1) Use the higher-level APIs (MediaPlayer, MediaElement)

    2) Use DirectShow + HwndHost

    Tuesday, September 26, 2006 9:04 PM
    Moderator
  • So I finally got time this weekend to sit down and play with the webcam preview (after getting tired of hearing KC nagging me about doing it) and lo and behold, I got it working.

    I must admit that the Directshow.Net library helped tremendously! Especially their wrapper around the SampleGrabber filter.

    SampleGrabber provides a callback that basically gives you access to the media sample as it goes through the graph. I hooked the SampleGrabber up to a video output pin and registered my callback.

    The callback uses BitmapSource.Create() to create a new bitmap from the current sample and assign it to a BitmapSource dependency property that is bound to an Image on a panel.

    Everytime a new sample comes in, it gets put on the panel. It appears to run as fast as a standard windows webcam preview app...but I can always get some metrics to check Frames Per second and stuff like that.

    Because all WPF sees is an image, I get to do cool stuff like transparency effects and layering, scaling, transforms...all of that good stuff. Of course it won't work in a sandboxed environment because of the unmanaged calls. But it will allow you to do something like put a channel from your tv tuner on the sidebar.

    Like I said before...fun little weekend project

     

    Monday, October 09, 2006 4:28 PM
    Moderator
  • awesome! can't wait to see it.

    casey

     

    Monday, October 09, 2006 7:31 PM
  • Hi Mike,

    Any chance of releasing the source-code for your findings?

    Thx,

    Waseem
    Monday, October 09, 2006 9:16 PM
  • The DxWebCam sample that comes with the samples accociated with the directshow.net library will be a good starting point. Once you uncompress the samples you can find the DxWebCam sample in ../samples/Misc. Here is the sample description

    "A poor man's web cam program. This application runs as a Win32 Service. 
    It takes the output of a capture graph, turns it into a stream of JPEG
    files, and sends it thru TCP/IP to a client application."

    The client application is a WinForms application. One can use WPF for the client app easily as the images from the web cam are bmp. And WCF can be used instead of the raw TCP/IP if needed. I hope this helps.

    -Ahmed
    Sunday, October 15, 2006 6:55 PM
  • I've already made the solution, but I didn't even realize there was a Webcam sample already. I'm using the SampleGrabber to push the samples onto a panel using the BitmapSource.Create() signature that takes an unmanaged pointer to a bitmap array.

    I promise once I get a chance to breathe (probably november), I'll get the information to you guys!

    Monday, October 16, 2006 9:24 PM
    Moderator
  • Hi ivolved_Mike_Brown,

        I have also been trying to do the same, but somehow I am stuck because the callback function and the main function seems to be running on two different threads, and I cannot access the main function item (a Image element) from my callback function. Sorry if this sound more like a basic programming problem, coz I am still new to C#. The code below shows my implementation at the callback function, where img is a image on the main panel.

           unsafe int ISampleGrabberCB.BufferCB(double SampleTime, IntPtr pBuffer, int BufferLen)
            {
                byte* m_buffer = (byte*)pBuffer;
            
                img.Source = null;

                img.Source = BitmapSource.Create(
                     m_videoWidth,
                     m_videoHeight,
                     96,
                     96,
                     PixelFormats.Bgr24,
                     null,
                     (IntPtr)m_buffer,
                     m_ImageSize,
                     m_stride);

                return 0;
            }

    Can point out to me what changes I need in order to let it perform correctly? Thanks
    Wednesday, October 25, 2006 8:03 AM
  • Yes you have to use the dispatcher to make a UI change from a different thread.

     unsafe int ISampleGrabberCB.BufferCB(double SampleTime, IntPtr pBuffer, int BufferLen)
            {
                byte* m_buffer = (byte*)pBuffer;
                img.Dispatcher.BeginInvoke(DispatcherPriority.Render,
                delegate
                {

                   img.Source = null;
                   img.Source = BitmapSource.Create(
                     m_videoWidth,
                     m_videoHeight,
                     96,
                     96,
                     PixelFormats.Bgr24,
                     null,
                     (IntPtr)m_buffer,
                     m_ImageSize,
                     m_stride);
                })
                return 0;
            }

    We got the Dispatcher from the Image control and called it's BeginInvoke method (this allows the function to return without waiting for the dispatcher to finish), and pass it an anonymous delegate (more performant than declaring a delegate and passing parameters to it) that does the work.

    Wednesday, October 25, 2006 2:41 PM
    Moderator
  • Thanks, that solves my problem!
    Thursday, October 26, 2006 4:06 AM