none
Memory issues with System.Drawing.Bitmap RRS feed

  • Question

  • I'm working on part of an application that handles image editing. The primary reason I'm looking at it is because it constantly runs out of memory. The application was originally quite poorly written and Bitmaps were never ever disposed of. So I went through every one and made sure the application always disposed of them properly. The way the application works is that (unlike regular image editors, like Photoshop or Paint.Net for example) is that adding effects (such as sharpen, blur, contrast, etc) is not cumulative - its applied based on the base image. So we keep in memory the base image, a working image, throughout the window's lifetime, reloading the base image to the workign image before re-applying the effects. In addition we also use WPF (hosted in a Winforms control) to display the image (we used to create a new one from the Bitmap's handle every time, but now I use a WritableBitmap, so I can create and reuse without worrying about the old one sticking around too long). And finally if an image is resized we run a "ImageProcessor" which applies all the effects and crops and saves it to a memory stream so we can "estimate" the file size. 

    Prior to my changes, we could load a 10MP image and apply an effect twice and it would run out of memory. Now we can edit "infinitely" and not run out of memory, as I've tried to reduce the memory foot print. The problem is that very rarely we still run out of memory. The confusing this is that I can't quite see how. 

    As part of working on this I wrote a little side app - it basically runs parallel to the main app and keeps track of its memory usage using performance counters. I get the ("Process", "Private Bytes") to give me the total memory allocated to the app. I use (".NET CLR Memory", "# Bytes in all Heaps") to see how much CLR memory is allocated. By doing PrivateBytes-CLRMemory I can see how much memory is used in CLR Managed memory and in Non-CLR unmanaged heap every second. 

    So using this, I get the following values when loading a bitmap(the memory is NON-CLR/CLR in MB as reported by my Performance counters)

    Load file into Byte Array - 103/87
    Load byte array into Base Image 157/91
    Load base image into Working Image - 204/91
    Create WriteableBitmap - 245/91
    **Loading calls finished and WritableBitmap Displayed** - 273/91  *The is the memory usage when running and not at a debug breakpoint
    Estimate File Size - 310/91
    Estimate File Size finished - 273/91

    So once a large Bitmap is loaded and displayed I'm basically looking at 273MB of constant memory usage with an additional 40-45MB for the estimation. 

    So I'm assuming that a Bitmap is a wrapper around an HBITMAP, and that the 10MP image is stored "uncompressed" as the raw RGB(A?) data. That would certainly fit my estimations above - a 10MP image is around 30MB in RGB space just in pure byte form. 

    The problem I'm having is that sometime the File Size Estimation fails with an OutOfMemory exception. Memory usage is no different to that above, and its throwing the OOM reporting NONCLR/CLR 275/91. 

    I'm on a machine with 4GB RAM (only 32bit tho of course, so only 3 and a bit GB is available), and the process is nowhere near the 2GB limit (its totalling around 365MB total), with a PerformanceCounter reporting that ("Memory", "Available Bytes") for the sytem is around 1.7-2GB Free. I'm pretty sure that I read that an HBITMAP requires continuous memory, but obviously thats out of my control. 

    I also have around 40GB free on my swap file drive, with the swap file size is system set, so should be able to go up to the full 40GB I assume.

    Is there any way to figure out why I'm running out of memory when I'd appear to have lots of memory spare?
    Monday, July 20, 2009 3:22 PM

Answers

  • Memory issues with System.Drawing.Bitmap or WritableBitmap

    Using WPF throughout should help.  64 bit OS helps immensly.  Latest ServicePacks help.  .NET GDI+ is not the best engine for image editing.  There are many open source engines that work well.
    • Marked as answer by thedo Tuesday, July 21, 2009 7:58 AM
    Monday, July 20, 2009 3:42 PM
  • You've got a problem with virtual memory address space fragmentation.  It is a common scourge on 32-bit operating systems when working with large bitmaps.  The problem is that the process is running out of contiguous mapable memory pages.  You can never get more than about 500 MB when you first start out, that starts dropping quickly once you start using memory.  You'll get OOM well before you've used up all virtual memory, regardless how much RAM or swap file space you've got.

    There isn't anything you can do about it.  It is otherwise a solved problem that takes 200 bucks.  64-bit operating systems have gobs of virtual address space.  Verify your memory diagnostics with SysInternals' VMMap utility.  Looking at workingset size is a common mistake.  You should be able to stay out of trouble with 40 MB bitmaps.

    Hans Passant.
    • Marked as answer by thedo Tuesday, July 21, 2009 7:59 AM
    Monday, July 20, 2009 3:57 PM
    Moderator

All replies

  • Memory issues with System.Drawing.Bitmap or WritableBitmap

    Using WPF throughout should help.  64 bit OS helps immensly.  Latest ServicePacks help.  .NET GDI+ is not the best engine for image editing.  There are many open source engines that work well.
    • Marked as answer by thedo Tuesday, July 21, 2009 7:58 AM
    Monday, July 20, 2009 3:42 PM
  • You've got a problem with virtual memory address space fragmentation.  It is a common scourge on 32-bit operating systems when working with large bitmaps.  The problem is that the process is running out of contiguous mapable memory pages.  You can never get more than about 500 MB when you first start out, that starts dropping quickly once you start using memory.  You'll get OOM well before you've used up all virtual memory, regardless how much RAM or swap file space you've got.

    There isn't anything you can do about it.  It is otherwise a solved problem that takes 200 bucks.  64-bit operating systems have gobs of virtual address space.  Verify your memory diagnostics with SysInternals' VMMap utility.  Looking at workingset size is a common mistake.  You should be able to stay out of trouble with 40 MB bitmaps.

    Hans Passant.
    • Marked as answer by thedo Tuesday, July 21, 2009 7:59 AM
    Monday, July 20, 2009 3:57 PM
    Moderator
  • Memory issues with System.Drawing.Bitmap or WritableBitmap

    Using WPF throughout should help.  64 bit OS helps immensly.  Latest ServicePacks help.  .NET GDI+ is not the best engine for image editing.  There are many open source engines that work well.

    We're actually using the Aurigma Library for processing, but a look around reflector looks like its still just a wrapper around a HBITMAP. 

    64bit is out of the question - none of our users (or the developers!) have 64bit CPUs, let alone 64bit OSes :( 

    All up to date on the service packs. 

    We're actually having our users beta-test a pure WPF version, but the Winforms/WPF hybrid will be used for at least another 8-12 months, so ideally we'd like to improve it for the rest of the products lifespan.


    You've got a problem with virtual memory address space fragmentation. It is a common scourge on 32-bit operating systems when working with large bitmaps. The problem is that the process is running out of contiguous mapable memory pages. You can never get more than about 500 MB when you first start out, that starts dropping quickly once you start using memory. You'll get OOM well before you've used up all virtual memory, regardless how much RAM or swap file space you've got.

    There isn't anything you can do about it. It is otherwise a solved problem that takes 200 bucks. 64-bit operating systems have gobs of virtual address space. Verify your memory diagnostics with SysInternals' VMMap utility. Looking at workingset size is a common mistake. You should be able to stay out of trouble with 40 MB bitmaps.
    Interesting stuff. I'd figured out that it was fragmentation, but was hoping there was some remedy that would help manage it more effectively. For some reason VMMap is crashing on attaching to the process, so I cant tell for 100% certainty, but thats what I suspected, and you seem to confirm. The situation with simply managing memory better has improved the situation ten-fold here, but I really wanted to be be able to solve the issue, rather than simply improve it - such is life. 

    Thanks for the thoughts guys.
    Tuesday, July 21, 2009 7:58 AM