none
How can I display live video in C# without any freezing or delay

    Frage

  • Im using a GigE camera from AVT. To use it in C# I import the methods from a dll that comes with the SDK of the camera.

    When one frame is grabbed, a callback method is called with a InPtr containing the frame data as argument.

    Actualy I'm creating a Bitmap and displaying it in a PictureBox. But deppending the frame rate, camera image size and PictureBox size, the software starts to freezes becouse the thread that displays the image eats up all process resources.

    To overcome that I limit the display refresh rate (about 30 FPS max). So if the camera FPS is higher than 30, the PictureBox does not display all income images.

    Another problem is that there is a delay in the displayed images. I can realize that by moving my hand in the front of the camera. It probably comes from the time waste building the bitmap and refreshing the picturebox.

    How can I improve my app performance? Do I need to quit C# and go for C++? :/

    C# is not capable off display live images without delay and freezing interface?

    DirectShow can do something for my problem?

    I'm open for suggestions. Any help will be appreciated! Thanks!

    Donnerstag, 21. April 2011 13:12

Alle Antworten

  • If you are using the following ctor:

    Bitmap(int width, int height, int stride, PixelFormat format, IntPtr scan0)

    then building the bitmap is a very lightweight operation, otherwise it can be more or less inefficient when wrapping a native buffer.

    Bitmap itself uses GDI+, which uses DirectDraw, so it is quite fast. But I believe the real bottleneck here is the PictureBox, which is designed for static images, not live video. You should draw the images yourself using System.Drawing.Graphics.

     


    MVP :: DirectShow / MediaFoundation <http://www.riseoftheants.com/mmx/faq.htm>
    Donnerstag, 21. April 2011 17:03
  • Actualy I do Bitmap(int width, int height, int stride, PixelFormat format, IntPtr scan0).

    I think you are right, can you give me a sample code using the System.Drawing.Graphics ?

    Thanks ! :)

    Donnerstag, 21. April 2011 18:51
  • I think you are right, can you give me a sample code using the System.Drawing.Graphics ?


    I don't use .NET.
    MVP :: DirectShow / MediaFoundation <http://www.riseoftheants.com/mmx/faq.htm>
    Donnerstag, 21. April 2011 20:47
  • I never liked PictureBox and never used it for video preview. You may actually first try it using a Panel. If that doesn't work for you, you may try to subclass Panel, override OnPaint event and draw bitmap there:

    protected override void OnPaint(PaintEventArgs e)
    {
      if(this.bitmap != null && this.bitmapLocation != Point.Empty)
      {
       e.Graphics.DrawImage(this.bitmap , this.bitmapLocation);
    } }

    where this.bitmap is the image you should draw.

    Off course, for more robust scenario, you may want to queue Bitmaps in a list, together with their presentation time and draw them (or drop them) according to that time.

    Donnerstag, 21. April 2011 21:02
  • Why you dont want the (0,0) point for the bitmap location? It is the uper left corner.

    I dont see why I need to subclass, I can add this code to the picturebox on paint event.

    I can use a BlockingCollection (.NET 4.0 and threadsafe) to enqueue the bitmaps and a foreach to consume it.

    I was using the picturebox on zoom mode, so the image have the size of the picrebox without loosing aspect ratio. Do you know an easy away to get the upper-left and botton-right corner of the picturebox image, so I can use:

    e.Graphics.DrawImage(this.bmp, new Rectangle(Poin1, Point2));

    and draw the image with correct aspect ration, size and location (centered).

     

    I have tried the DrawImage  but it is a lot slower than setting the picturebox image and refreshing it. With a 800x600 image, drawing in almost full screen 1920x1080 takes about 25 ms (not using the correct aspect ratio)!!! :/

    -----------------------

    The reason why: http://www.bobpowell.net/why_so_slow.htm

    Why so slow?

    Anyone who has tried a fairly graphic intensive application with GDI+ will have noticed that it's just a little slow when compared with older GDI graphics operations.

    The reason for this is that GDI+ drivers do not, as yet, take advantage of hardware acceleration features built in to many of the graphics cards fitted to PC systems today.

    Later versions of GDI+ will have hardware acceleration enabled.

    For seriously speedy .NET graphics applications, check out DirectX9 (directx.asp)

    ----------------------

    Use DirectX ?? :(

    Montag, 25. April 2011 12:10
  • DrawImage can't be slower then setting a static image to PictureBox (at least if you're not doing any resizing of the image in the Draw method). Try using DrawImageUnscaled function.

    As Allessandro noted earlier, PictureBox is designed to accommodate static images and had not been designed to be used in this way. That's why I recommended using something like Panel (that doesn't contain any code for handling images so you would do everything on your own.)

    If you have a Bitmap (or an Image) you already have the height and width of the image. The location of the image can be calculated depending on the size of the control you're drawing onto and the size of the image.

    Once again, if the drawing is slow then make sure you don't have any resizing going on anywhere. However, if this is requirement, then maybe you should try Direct3D.

     

    Montag, 25. April 2011 20:21
  • The control size avaliable to display the image is not necessary the same of the image, so resizing is requirement, and in this case the picturebox is much faster than DrawImage.

    Direct3d? Any sample? :D

     

    Montag, 25. April 2011 20:36
  • AFAIK, the fastest way to push pixels on the window (without doing Direct3D) is BitBlt (or, in your case  StrechBlt function). You can P/Invoke them (this  is p/invoke of bitblt, you should do strechblt yourself).

    About Direct3D - it shouldn't be that hard. You just need to Create Device, load the images on textures and present them. You may want to download DirectX SDK and check the samples.

    Montag, 25. April 2011 21:04
  • The camera comes with a C++ viewr. Its display the image with no delay, uses 0% of cpu and 11.7 kb of memory. My app in C# uses 8% CPU (deppending the windows size), 90 kb memory and does have delay!! Very poor performance :/

    Ill try BitBlt and then Direct3D

    Donnerstag, 28. April 2011 14:35
  • Hi PedroDD,

    I have the same problem. Also I've tried display images using the PictureBox or using the DrawImage function. Also I need image resizing. But both are too slow for resizing a real time video. How did you solve it? Could you give me any advice? Thanks you very much in advance.

    Donnerstag, 21. Juni 2012 10:28
  • Hi Jaromir !

    Unfortunately I did not found any good solution up to now.
    I'll try to use WPF, but at the moment I'm solving another problems. After I try this another approach I will post the result here.

    Another solutions that you can try would be directshow and maybe EmguCV (OpenCV) that can have some UI optimized methods to do that.
    --------
    I have just researched about it (camera video C#) at SO and found this thread:
    http://stackoverflow.com/questions/1667478/what-is-the-easiest-fastest-way-to-capture-video-stream-from-camera-with-c

    Maybe it is our solution. Please update here with anything you try! :)

    Donnerstag, 21. Juni 2012 12:43
  • Hi Pedro,

    finally I can post some results. Unfortunately I didn't try your link. I'm not sure if it could help. Probably the DirectShow could be good solution but I don't have any experiences with using DirectShow. Have you tried it?

    I have used Emgu CV. There is a ImageBox user control that is similar to PictureBox. But instead of displaying Bitmap, it displays Image<,> object. So you have to convert your bitmap to the Image<,> object. In my case it doesn't work. I don't know why. Maybe I used wrong image type.

    However, I found out that there is also a 'panAndZoomPictureBox'. Using of this control is the same like using PictureBox. So you don't have to convert anything. In addition, the panAndZoomPictureBox provides functionality for zooming the image. It works very well and without any delay. You can try it.

    Mittwoch, 8. August 2012 14:00
  • Im familiar to EmguCV, but Ive never used its ImageBox.

    How you are updating your panAndZoomPicuteBox? Do you update it every time one new image is grabbed? What is the acquisition frequency?

    If I update the screen at 30 FPS with an 1024 x 1024 image and using the software maximized, the others buttons dont update. Only the picturebox.

    Mittwoch, 8. August 2012 16:14
  • Actually the panAndZoomPictureBox is updated every time one new image is grabbed. But I'd like to limit the display refresh rate. How did you limit it?

    The acquisition frequency depends on the camera which I'm using and also on the exposition time. I tested a camera with 30fps (1296 x 966) but also a camera with 100fps (659 x 494). Size of my panAndZoomPictureBox was approximately 1600 x 900.

    About updating:

    "If I update the screen at 30 FPS with an 1024 x 1024 image and using the software maximized, the others buttons don't update. Only the picturebox."

    1024 x 1024 is it the display resolution? And which buttons do you mean?

    Do you know if there any possibility how to disable the tool for selecting a part of the image in the panAndZoomPictureBox? I need just the zoom function without the possibility to select a part of the image.


    • Bearbeitet jaromirJZ Donnerstag, 30. August 2012 07:52
    Mittwoch, 8. August 2012 19:32
  • Use a timer to check if some delay have passed.

    ex:

    Start the timer when you start the camera

    if (timer.Elapsed > minDelay)
    {

    Display image..

    timer.Reset();

    }

    The rest of the interface is not freezing ? Im mean anything other than the imageBox is freezes, does not refreseh properly when acquiring.

    Mittwoch, 8. August 2012 20:05
  • Sorry for late reply. I don't know if I understand your solution correctly. But, I think that the timer is not very reliable. Especially if your application is overloaded. I'm using combination of 'DateTime' and 'TimeSpan' to check if a delay have passed. But, I'm not sure if it is a best solution.

    And about freezing:
    When I used a PictureBox the situation was exactly the same as you say. The rest of interface wasn't able to work correctly if the image was zoomed. But when I'm using  panAndZoomPictureBox the rest of my interface works well also if the image is zoomed.
    How do you refresh your PictureBox? Do you use the 'pictureBox.Refresh();' function? If yes, try instead of it following: 'pictureBox.Image = your_bitmap;'. It's much faster.

    If you are familiar with EmguCV, do you know how to disable the function for selecting a part of the image in the panAndZoomPictureBox?


    • Bearbeitet jaromirJZ Donnerstag, 30. August 2012 07:49
    Donnerstag, 23. August 2012 12:18
  • Any idea? Have you tried anything?
    Donnerstag, 30. August 2012 07:51
  • The stopwatch can be reliable, it depends what you want to measure. Actualy the Stopwatch class is your best possible solution to measure time internally.

    From msdn: http://msdn.microsoft.com/en-us/library/system.diagnostics.stopwatch.aspx

    "The Stopwatch measures elapsed time by counting timer ticks in the underlying timer mechanism. If the installed hardware and operating system support a high-resolution performance counter, then the Stopwatch class uses that counter to measure elapsed time. Otherwise, the Stopwatch class uses the system timer to measure elapsed time. Use the Frequency and IsHighResolution fields to determine the precision and resolution of the Stopwatch timing implementation."

    Becouse we are measuring like hundreds of milisecounds, Stopwatch or DateTime will work ok. What delay are you using?

    It is very interresing that the panAndZoomPictureBox is not freezing your interface, I will try it too.

    About the refresh, actually when you change an Image of the picturebox it will get Invalidated (same of calling .Invalidate()) and the UI will redraw on the next update. When using Refresh you force the UI to redraw the control NOW.

    Im using the picturebox.Image = my_bmp. and leting the UI update when it wants (not calling Refresh)..

    Are you using another thread to capture the images and update the UI? I'm doing it. So I'm using the BeginInvoke to update the UI without cross-thread problem.

    I want to try to use directshow in WPF, but unfortunately now I do not have time.. :(



    • Bearbeitet PedroDD Donnerstag, 30. August 2012 13:55
    Donnerstag, 30. August 2012 13:50
  • Hi Pedro. Sorry for very late answer. I had a lot of other projects and with holiday...

    About your question: Yes, I am using another thread to capture the images as you do.

    Now I have done more tests and I found out that my program works well with cameras up to 2MPx. When I use cameras with resolution 2MPx and more it seems that the program is overloaded. Using 5MPx camera sometimes causes problems with UI. But I think that it is rather hardware problem.

    Have you tried the DirectShow?
    • Bearbeitet jaromirJZ Mittwoch, 7. November 2012 09:57
    Donnerstag, 11. Oktober 2012 07:40
  • Oww 5 MP, what cameras are you using?

    Are you displaying text info like: requested (or camera) FPS and displayed FPS?

    I haven't tried the DirectShow yet.

    Dienstag, 23. Oktober 2012 11:55
  • I use Basler cameras. I have tested a lot of different types. Specifically, the 5MPx camera which I have tested was Basler piA2400-17gc. I am displaying both, camera fps and displayed fps.

    Mittwoch, 7. November 2012 10:20
  • Hello Jaromir

    Are you displaying the FPS info in a textbox? How ofen do you updated it?

    One thing that I need to do is to call Refresh() to the entire form some times. I do it to be able to click the buttons, if I don't do it, only the image and fps info will be updated and the rest of the interface will get freezed.

    Mittwoch, 7. November 2012 10:45