none
[UWP] Bitmap class, access raw pixel data

    Question

  • Hi,

    I am currently migrating some functionality to a new UWP app. It seems that in the UWP framework the System.Drawing.Bitmap class

    is missing. Searching around, I found suggestions to use SoftwareBitmap with WriteableBitmap classes or the NuGet

    package WriteableBitmapEx.

    In SoftwareBitmap and WriteableBitmap I could not find access to the Pixel values (something corresponding to Bitmap.SetPixel).

    Going through the examples of WriteableBitmapEx I am missing some methods (like BitmapFactory.New().FromResource()).

    Can anyone give me a short concrete example on how to do the following:

    Load a local bitmap image (maybe also changing its dimensions), accessing pixel values (get and set) and saving the modified image as bitmap?

    regards,

    Max

    Thursday, October 6, 2016 4:36 PM

Answers

  • How is the byte[] sourcePixels used? I expected a two-dimensional Array corresponding to the image dimensions.

    WriteableBitmap provides the low level buffer containing ARGB pixel data. You can calculate a pixel's position in the array given it's stride. If the bitmap is W pixels wide, then its stride S is W*4, and pixel X,Y is the four bytes (ARGB) starting at X*S+Y . See the WriteableBitmap.PixelBuffer documentation and samples.

    If you want higher level accessors then you'll need to use something like the (third party) WriteableBitmapEx you mentioned in your initial post.

    And why is WriteableBitmap.GetPixel so slow?

    I assume you mean WriteableBitmapEx's GetPixel. WriteableBitmap doesn't have a GetPixel method.

    It sounds like you're not keeping the bitmap context between calls, so every call to GetPixel reads out the entire buffer, finds the pixel, and then discards the buffer. Instead, get the context once (which reads out the buffer) and keep it so that subsequent GetPixel calls will act on the existing buffer. This is explained and demonstrated in the WriteableBitmapEx page at https://github.com/teichgraf/WriteableBitmapEx/ and http://kodierer.blogspot.de/2012/05/one-bitmap-to-rule-them-all.html .


    Tuesday, October 11, 2016 4:55 PM
    Owner

All replies

  • Hi MaxStricker,

    As far as I know, you could access the pixels of WriteableBitmap by PixelBuffer. In C#, use AsStream extension method to access the underlying buffer as a stream.

    //get pixels from WriteableBitmap using (Stream stream = WriteableBitmap.PixelBuffer.AsStream()) { using(MemoryStream memoryStream = new MemoryStream()) { await Stream.CopyToAsync(memoryStream); byte[] pixels = memoryStream.ToArray(); } }

    //set pixels to WriteableBitmap
    using (MemoryStream memoryStream = new MemoryStream(pixels))
    {
            using(Stream stream = WriteableBitmap.PixelBuffer.AsStream())
            {
                    await memoryStream.CopyToAsync(stream);
            }
    }

    And use CopyToBuffer and CopyFromBuffer to transfer pixels between SoftwareBitmap and PixelBuffer.

    You could also access pixels of local bitmap files by BitmapEncoder and BitmapDecoder. In three steps, get the IRandomAccessStream of the files,  pass to the CreateAsync methods and call SetPixelData/GetPixelDataAsync.

    //get pixels from local file
    using (IRandomAccessStream fileStream = await file.OpenAsync(Windows.Storage.FileAccessMode.Read)) 
    {
        BitmapDecoder decoder = await BitmapDecoder.CreateAsync(fileStream); 
        // Scale image to appropriate size 
        BitmapTransform transform = new BitmapTransform() {  
            ScaledWidth = Convert.ToUInt32(WriteableBitmap.PixelWidth), 
            ScaledHeight = Convert.ToUInt32(WriteableBitmap.PixelHeight)
        }; 
        PixelDataProvider pixelData = await decoder.GetPixelDataAsync( 
            BitmapPixelFormat.Bgra8, // WriteableBitmap uses BGRA format 
            BitmapAlphaMode.Ignore, 
            transform, 
            ExifOrientationMode.IgnoreExifOrientation, // This sample ignores Exif orientation 
            ColorManagementMode.DoNotColorManage
        ); 
     
        // An array containing the decoded image data, which could be modified before being displayed 
    byte[] sourcePixels = pixelData.DetachPixelData();
    }
    //set pixels and save image to local file
    using (IRandomAccessStream fileStream = await file.OpenAsync(FileAccessMode.Read))
    {
            BitmapEncoder encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.BmpEncoderId, fileStream);
            encoder.SetPixelData(BitmapPixelFormat.Bgra8, BitmapAlphaMode.Ignore, (uint)image.PixelWith, (uint)image.PixelHeight, 100, 100, pixels);
            await encoder.FlushAsync();
    }
    Please also refer to the link below for more information about how to create, edit and save bitmap images for UWP apps.
    https://msdn.microsoft.com/windows/uwp/audio-video-camera/imaging

    Best Regards,
    David


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place. Click HERE to participate the survey.


    Friday, October 7, 2016 9:25 AM
    Moderator
  • How is the byte[] sourcePixels used? I expected a two-dimensional Array corresponding to the image dimensions.

    I need to replicate this two snippets (independently). I suspected that it might get tricky, but I could not find any solution.

    And why is WriteableBitmap.GetPixel so slow? I need 30-40sec to loop through a 5123x512 bitmap file, whereas in a Windows Forms App it used to be instant.

    Bitmap initial = new Bitmap(1, 1);
    initial.SetPixel(0, 0, Color.White);
    Bitmap image = new Bitmap(initial, IMAGE_WIDTH, IMAGE_HEIGHT);
    for (int i = 0; i < IMAGE_WIDTH; i++) {
        for (int j = 0; j<IMAGE_HEIGHT; j++) {
            if (i % 2 == 0) image.SetPixel(i, j, Color.Black);
        }
    }
    image.Save(FILENAME);
    
    
    //functionality two
    Bitmap image = new Bitmap(FILENAME);
    for (int x = 0; x < image.Width; x++)
    {
        for (int y = 0; y < (image.Height/8); y++)
        {
    		Color pixel = image.GetPixel(x, y);
            if (pixel.GetBrightness() < 0.5){
    			//do something
    		}
        }
    }

    Tuesday, October 11, 2016 4:11 PM
  • How is the byte[] sourcePixels used? I expected a two-dimensional Array corresponding to the image dimensions.

    WriteableBitmap provides the low level buffer containing ARGB pixel data. You can calculate a pixel's position in the array given it's stride. If the bitmap is W pixels wide, then its stride S is W*4, and pixel X,Y is the four bytes (ARGB) starting at X*S+Y . See the WriteableBitmap.PixelBuffer documentation and samples.

    If you want higher level accessors then you'll need to use something like the (third party) WriteableBitmapEx you mentioned in your initial post.

    And why is WriteableBitmap.GetPixel so slow?

    I assume you mean WriteableBitmapEx's GetPixel. WriteableBitmap doesn't have a GetPixel method.

    It sounds like you're not keeping the bitmap context between calls, so every call to GetPixel reads out the entire buffer, finds the pixel, and then discards the buffer. Instead, get the context once (which reads out the buffer) and keep it so that subsequent GetPixel calls will act on the existing buffer. This is explained and demonstrated in the WriteableBitmapEx page at https://github.com/teichgraf/WriteableBitmapEx/ and http://kodierer.blogspot.de/2012/05/one-bitmap-to-rule-them-all.html .


    Tuesday, October 11, 2016 4:55 PM
    Owner
  • Thanks for the clarification. I have still no working solution and could need some guidance.
    My current solution is using WriteableBitmapEx but any other solution is fine for me.
    This is my current idea:

     private async void processMessage(){
     WriteableBitmap writeableBmp = BitmapFactory.New(128, 128);
        using (writeableBmp.GetBitmapContext())
        {
            writeableBmp.Clear(Colors.White);
            
            // processing message and repeatedly perform 
            writeableBmp.SetPixel(x, y, color);
            
            WriteableBitmap resized = writeableBmp.Resize(512, 512, WriteableBitmapExtensions.Interpolation.Bilinear);
            String fileName = Guid.NewGuid().ToString().Replace("-", "") + ".bmp";
            Debug.WriteLine(fileName);
            var outputFile = await ApplicationData.Current.LocalFolder.CreateFileAsync(fileName, CreationCollisionOption.ReplaceExisting);
            using (var writeStream = await outputFile.OpenAsync(FileAccessMode.ReadWrite))
            {
                await EncodeWriteableBitmap(resized, writeStream, BitmapEncoder.JpegEncoderId);
            }
        }
    }
     I am getting "Exception from HRESULT: 0x8001010E (RPC_E_WRONG_THREAD)" when calling WriteableBitmap's constructor. It seems to  me that WriteableBitmap requires the UI thread to work. I am calling this method when receiving data from a Socket which runs inside a 
    Task.Run(async () => {});
    therefore the UI thread is not available. Looking for solutions, it seems that UWP has removed all classes used in possible solutions to this problem.
    So, I have to reformulate my original question: How can I read and write a bitmap file by accessing its raw pixels in a background thread using UWP?

    Any suggestions? This is a show-stopper at the moment.


    Thursday, October 13, 2016 1:57 PM
  • That's enough of a reformulation to be an entirely different question. WritableBitmap and other classes designed to display images are the wrong tools. To encode and decode image files use the BimapEncoder and BitmapDecoder classes. See their MSDN pages for examples and links to full samples.
    Thursday, October 13, 2016 3:03 PM
    Owner
  • @RobCaplan - When you said "you can calculate a pixel's position in the array given it's stride. If the bitmap is W pixels wide, then its stride S is W*4, and pixel X,Y is the four bytes (ARGB) starting at X*S+Y" I believe this is slightly wrong.

    The slight omission is that we need to take into account that byte arrays start at zero so X*S+Y will not work if we want to get the last pixel on a row or column. For example, if you had a 100 x 100 image then the last pixel on each line does not at start at byte 400, it starts at byte 396. And obviously the same applies to the rows.  So the equation is (Y * (imageWidth * 4)) + (X * 4) where X or Y  has one subtracted from it if the image width or height is passed in as X or Y respectively. 

    Here's some code I wrote to get the colour for any pixel in the image:

    pubilc Color GetPixel(byte[] pixels, int x, int y, int imageWidth, int imageHeight)
            {
                //If either x = image width or y = image height we need to subtract 1 due to byte arrays starting at 0.
                // e.g: for a 512 x 256 image, if the x parameter is 512 then x would become 511.
                //Obviously the same concept applies to the rows as well - row 256 is actually 255.
                if (x == imageWidth)
                    x = imageWidth - 1;
                if (y == imageHeight)
                    y = imageHeight - 1;

                //The following equation can be written all on one line (as below) but I've expanded it here so it's easier to understand.
                //int k = (y * (imageWidth * 4)) + (x * 4);
                int rowWidth = imageWidth * 4;          
                int rowOffset = y * rowWidth;
                int colOffset = x * 4; 
               int k = rowOffset + colOffset;

                //Note the pixel colour order is *not* ARGB, it is actually BGRA
                var b = pixels[k + 0];
                var g = pixels[k + 1];
                var r = pixels[k + 2];
                var a = pixels[k + 3];

                Color pixelColor = Color.FromArgb(a, r, g, b);
                if (a == 0 && (b + g + r == 0))
                    return Colors.Transparent;
                else
                    return pixelColor;
            }

    I'm not posting this to necessarily correct you Rob, it's more to help others to avoid going through what I had to in order to get the pixel colour anywhere in an image... it took me days (and much gnashing of teeth and pulling of hair) to work out how to do it using BitmapDecoder, WriteableBitmap and then getting the correct pixel using the code above. We use it in our UWP app to work out the background colour to show for a user provided image from the  pixel at each corner of the image.








    • Edited by nzmike001 Wednesday, December 5, 2018 11:26 PM
    Wednesday, December 5, 2018 11:15 PM
  • Not quite. X and Y are already zero-indexed and don't need further adjustment. 

    X = imageWidth is out of bounds, not the right most column. The valid values for X are 0 through imageWidth-1. You can check in debug mode with asserts:

       Debug.Assert( 0 <= x && x < imageWidth,"x out of bounds");
       Debug.Assert( 0 <= y && y < imageHeight,"y out of bounds");
    

    By correcting x == imageWidth to x = imageWidth - 1 your code ends up with two values for the last column. Better would be to reject that as an invalid argument and fix the calling code to pass imageWidth-1 if it wants the last column.


    Thursday, December 6, 2018 10:36 PM
    Owner