This is a cross posting of my Stack Overflow question (here)
I found several patterns for optimizing Bitmaps handling in WPF. I do not, however, understand when to use each patterns. As I think this is a common problem, I summarized what I understand and what I guess and ask for your help. If you can add patterns, explain how they differ, explain if they use the CPU or the GPU, and teach when to use each and how to combine them, it’d be a tremendous help!
Context – the Images "Grid" Scenario:
My application has to display many bitmap images. The images are displayed on the screen in a rows-and-columns grid-like organization (not necessarily the Grid or UniformGrid classes, think Window Media Player’s Album view). Images might move between different grid cells. Some images at arbitrary cells may be replaced by others. Images should be clickable, should provide a context menu, should be selectable, drag-able etc. In other words, “combine the little buggers into one big bitmap” is not applicable, at least not naively.
Patern 0: The Hack
Do combine the little buggers into a bitmap (how? drawing context?), and use this as the backgroud. Overlay this with images with empty content that will handle the hits, context menus, events etc.
The advantage is that we're only speaking about two bitmaps here: The curently displayed one and the one that should replace it. This should be really fast. However, my years of experience raise the red flag of danger. Your comments?
Pattern 1: Reduce Image Size
This is a no-brainer when you know in advance the image size to resize to, and when you’re prepared to lose details (color) for performance:
- Reduce the bitmap size using BitmapImage.DecodePixelWidth
- Reduce the color information using FormatConvertedBitmap.DestinationFormat
- Set the control’s scaling behavior setting Image.Stretch to Stretch.None
- Set the SetBitmapScalingMode for the image to LowQuality.
- Freeze the bugger
See code here.
Pattern 2: Background pre-fetch
This pattern is applicable when you think you can take advantage of the user gazing at the images on the screen, and prepare ahead the next images to display. The cons for your project, in addition to the memory overhead, is that it has to support the .Net Framework 4 target and not just the client profile, so it might incur an installation on the client’s. You yourself will have to suffer the async programming pain.
In this pattern you create exactly the required number of Image controls. When bitmaps need to be added, moved or deleted you only modify the Image controls' BitmapSource(s). A BackgroundWorker task is responsible for pre-fetching the BitmapSource(s) (possibly using the “Reduce Image Size” pattern above) and and inserting them into MemoryCache.
For this to work you have to set the BitmapImage’s CacheOption to OnLoad, so that the work is off-loaded to the background worker.
Pattern 3: Drawing Context
This was suggested by Sheldon Ziao from Microsoft Support on the MSDN WPF forum here. See page 494, Chapter 15 “2D Graphics” in Adam Nathan’s WPF 4 Unleashed for a description of DrawingContext. I can’t say I understand it. According to the answer here, I would assume this would improve handling of Geometry drawings, not bitmaps. Next, I don’t think this will support the focus and events requirements for the images (my bad for not explaining the requirements better at the forum) Moreover, I’m worried by the book’s summary sentence: “Note that the use of DrawingContext doesn’t change the fact that you’re operating within a retained-mode system. The specified drawing doesn’t happen immediately; the commands are persisted by WPF until they are needed.” This means that once our even handler re we can’t take advantage of parallelism as in “Background pre-fetch”.
Pattern 4: Writeable Bitmaps
The MSDN documentation here describes it as a dual buffer system: Your UI thread updates the buffer; the WPF’s render thread moves this to video memory.
The intended usage (see here) is for bitmaps that change a lot such in a video movie like display. I’m not sure, but possible this could be hacked and combined with the Background Pre-fetch pattern and used in the grid scenario.
Pattern 5: Cached Bitmap
Not much info on the MSDN (here). On the WPF forum archive (here) it’s explained that “The BitmapCache API is designed to cache your content (when rendering in hardware) in video memory, meaning it stays resident on your GPU. This saves you the cost of re-rendering that content when drawing it to the screen.” This seems like a great idea. I’m not sure, however, what are the pitfalls and how to use it.
Pattern 6: RenderTargetBitmap
The RenderTargetBitmap converts a Visual to a bitmap. I’m not sure if it’s relevant here. See here.
Edit: Regarding Paul Hoenecke question: I've written that "My application has to display many bitmap iages". I failed to mentions that I need to display about 800 images concurrently.
One can read about the performance issues involved at my SO questions WPF Bitmap performance and How can I make displaying images on WPF more “snappy”?
I've modified the description of pattern 1 to highlight the concept that the image controls aren't created or deleted (unless we want to display a larger or smaller grid). Only their Sources are set to different, new or null BitmapSources.
Thank you for your question, I will involve someone familiar with this topic to further look at your issue.
MSDN Community Support | Feedback to us
Please remember to mark the replies as answers if they help and unmark them if they provide no help.
You have provided many bitmap related concept and I'm not familiar with everyone. Well, I will try to understand your scenario and show my suggestion.
Based on my understanding, you have one grid like control, and there are multiple bitmaps on the control. The bitmaps should support context menu, drag & drop etc.
I will suggest you use one listview, each listview item can host one image control, and its source is binded to the underling item struct; as far as you have defined your item template correctly, the bitmap will be scaled automatically. You might concern the memory usage of listview, because we can suppose too many bitmaps will cost bunch of memory. So, you can use VirtualingPanel as the listview's panel, also set the bitmap source as weakreference. Then, when the list view item is out of view box, the corresponding bitmap can be gc collected.
Let's get back to your questions about the usage of the above six patterns.
"Pattern 0: The hack" and "Pattern 3: Drawing Context" should not meet your requirement. Definitely, drawing context is able to paint bitmap, however, the bitmap's scale has to be controled by yourself, this is not a comfortable task. Also, you have to handle context manually for different click position.
"Pattern 4: Writeable Bitmap" doesn't meet your requirement. You just need to show the bitmap, don't need to change the pixels.
"Pattern 6: RenderTargetBitmap" is used to render one visual to a bitmap, such as from one button to a bitmap, nothing to do with your requirement.
I don't know "Pattern 5: Cached Bitmap"
I doesn't quite understand your meaning about "Pattern 2: Background pre-fetch". But i think it is good idea and maybe you have to use background worker to fetch each bitmap source, when the bitmap source is loaded, set it into weakreference and fire property changed event. Otherwise, you app might suffer load performance issue.
Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread.
Microsoft Online Community Support
Thank you for your answers. I do not feel, however, that I do understand the recommended approach to take. Could someone from the WPF team, who is familiar with the WPF internals and its interaction with lower level display systems, take a look at this question?