locked
Bitmap transformations produce incorrect output for some images

    Question

  • I am trying to scale a jpeg image to half, this is the small code I am using:

    void GenerateThumbnail(StorageFile^ imageFile) 
    {                        	
            create_task(imageFile->OpenAsync(Windows::Storage::FileAccessMode::Read)).then([=] (Windows::Storage::Streams::IRandomAccessStream ^stream) 
    		{            
    			create_task(Windows::Graphics::Imaging::BitmapDecoder::CreateAsync(stream)).then([=] (Windows::Graphics::Imaging::BitmapDecoder ^decoder) 
    			{
    				auto transform = ref new Windows::Graphics::Imaging::BitmapTransform();
    				
    				transform->ScaledWidth = double(decoder->OrientedPixelWidth)/2;
    				transform->ScaledHeight = double(decoder->OrientedPixelHeight)/2;
    				
    				// The BitmapDecoder indicates what pixel format and alpha mode best match the
    				// natively stored image data. This can provide a performance and/or quality gain.
    				auto pixelFormat = decoder->BitmapPixelFormat;
    				auto alphaMode = decoder->BitmapAlphaMode;
    				auto dpiX = decoder->DpiX;
    				auto dpiY = decoder->DpiY;
    
    				// Get pixel data from the decoder. We apply the user-requested transforms on the
    				// decoded pixels to take advantage of potential optimizations in the decoder.
    				create_task(decoder->GetPixelDataAsync(
    					pixelFormat,
    					alphaMode,
    					transform,
    					Windows::Graphics::Imaging::ExifOrientationMode::RespectExifOrientation,
    					Windows::Graphics::Imaging::ColorManagementMode::ColorManageToSRgb
    					)).then([=] (Windows::Graphics::Imaging::PixelDataProvider^ pixelProvider)
    					{						
    						// Now that we have the pixel data, get the destination file
    						create_task(ApplicationData::Current->LocalFolder->CreateFolderAsync("test", CreationCollisionOption::OpenIfExists)).then([=](StorageFolder ^folder)
    						{							
    							create_task(folder->CreateFileAsync("testScaled.jpg", Windows::Storage::CreationCollisionOption::ReplaceExisting)).then([=] (task<Windows::Storage::StorageFile^> fileTask) 
    							{						
    								auto file = fileTask.get();
    								create_task(file->OpenAsync(Windows::Storage::FileAccessMode::ReadWrite)).then([=] (task<Windows::Storage::Streams::IRandomAccessStream^> streamTask) 
    								{								
    									auto stream = streamTask.get();	
    									stream->Size = 0;
    									create_task(Windows::Graphics::Imaging::BitmapEncoder::CreateAsync(Windows::Graphics::Imaging::BitmapEncoder::JpegEncoderId, stream)).then([=] (task<Windows::Graphics::Imaging::BitmapEncoder^> encoderTask) 
    									{
    										
    										auto encoder = encoderTask.get();										
    										auto pixels = pixelProvider->DetachPixelData();
    																					
    										encoder->SetPixelData(
    											pixelFormat,
    											alphaMode,
    											transform->ScaledWidth,
    											transform->ScaledHeight,
    											dpiX,
    											dpiY,
    											pixels
    											);									
    										create_task(encoder->FlushAsync()).then([=](task<void> flushTask)
    										{
    											flushTask.get();
    										});
    																				
    								});
    							});
    						});
    					});						
    				});
    			});
    		});
    }


    and, then I call this function like:

    FileOpenPicker^ openPicker = ref new FileOpenPicker();
            openPicker->ViewMode = PickerViewMode::Thumbnail;
            openPicker->SuggestedStartLocation = PickerLocationId::PicturesLibrary;
            openPicker->FileTypeFilter->Append(".jpg");
            openPicker->FileTypeFilter->Append(".jpeg");        

            create_task(openPicker->PickSingleFileAsync()).then([this](StorageFile^ file)
            {
                GenerateThumbnail(file);
            });

    The code works fine if I pass an empty transform and pass the full dimensions in setpixel. So, it is the issue with transform only.

    I think the issue appears for the images which have exiforientation flag set, but I am not sure.

    I have uploaded a blank sample app with this function to show the bug and the also the image with which it happens.

    I am clueless, I have checked all the function parameters and their values but there is no hint. Please help.




    • Edited by John Rick Tuesday, September 18, 2012 3:17 PM
    Tuesday, September 18, 2012 2:58 PM

Answers

  • Hi John,

    The problem is that you are rotating the image when you read it in, but not when you write it out: the value passed to the BitmapEncoder has the Height and Width reversed, so the pixels aren't copied to the correct locations. If you pass height and width based on decoder->PixelHeight and decoder->PixelWidth instead of based on OrientedPixelHeight and Width then it will save out correctly both for images which are rotated and for images which are not rotated.

    Using OrientedPixelHeight and OrientedPixelWidth are correct for the BitmapDecoder, but not the BitmapEncoder.

    --Rob

    Wednesday, September 19, 2012 1:55 AM
    Owner
  • Ok. I can reproduce this now with your picture. It looks like there may be a problem with the math when combining scaling and EXIF rotation, but you can avoid that by doing it yourself:

    • Check the orientation from imageFile->Properties->GetImagePropertiesAsync 
    • Set the transform's rotation based on the ImageProperties' Orientation.
    • Rotate the encoding height and width based on the new orientation
    • Set the ExifOrientationMode to IgnoreExifOrientation

    --Rob

    Friday, September 28, 2012 2:39 AM
    Owner

All replies

  • Hi John,

    The problem is that you are rotating the image when you read it in, but not when you write it out: the value passed to the BitmapEncoder has the Height and Width reversed, so the pixels aren't copied to the correct locations. If you pass height and width based on decoder->PixelHeight and decoder->PixelWidth instead of based on OrientedPixelHeight and Width then it will save out correctly both for images which are rotated and for images which are not rotated.

    Using OrientedPixelHeight and OrientedPixelWidth are correct for the BitmapDecoder, but not the BitmapEncoder.

    --Rob

    Wednesday, September 19, 2012 1:55 AM
    Owner
  • Thanks Rob,

    What is the use of passing RespectExifOrientation to decoder->GetPixelDataAsync then?

    I thought passing this flag will return me the rotated bitmap, am I wrong? How will I restore the original orientation in my destination image then?

    As illustration, say I have a image of pixels 640x480 so that its oriented dimensions are 480x640.

    So passing RespectExifOrientation to decoder->GetPixelDataAsync should give me the bitmap of 480x640 respecting the exif orientation. And if so, I passed the same values to encoder. What is the discrepancy? Please help.

    • Edited by John Rick Wednesday, September 19, 2012 2:46 PM
    Wednesday, September 19, 2012 2:42 PM
  • Hello Rob,

    As suggested by you, If I use the decoder->PixelHeight and decoder->PixelWidth, the image is awkwardly stretched.

    As I said in the previous reply, I doubt this was the actual problem. I don't understand why RespectExifOrientation will not be honored by the GetPixelDataAsync?

    Could you or someone please reply further? I am stuck here!!!

    Friday, September 21, 2012 7:14 AM
  • Can you please provide examples of your expected results and of your actual results (both original and awkwardly stretched)?

    The results I got with your original code looked like they had the right pixels, but in the wrong places: similar to what I expect when the stride is wrong. The output didn't look obviously stretched to me, so either I made different changes than you did or I didn't look at the image as closely.

    I don't have that available to me now, but I'll take a closer look when I am back in the office next week.

    --Rob

    Sunday, September 23, 2012 3:51 AM
    Owner
  • Hi Rob,

    The thumbnail of original image is this which has Pixel bounds (640x480) but oriented bounds become (480x640):

    Original Change

    After the changes you suggested, I passed decoder->PixelWidth/2 and decoder->PixelHeight/2 to the SetPixelData, and I got the following image:

    I still don't understand how your suggestion will work. Who on earth will take care of the orientation? Now the image dimension is written as 320 x 240 while it should 240 x 320.

    Monday, September 24, 2012 10:59 AM
  • The correct and expected image should look like this, with dimension 240 x 320:The correct and expected image is this:
    Monday, September 24, 2012 11:00 AM
  • Hi Rob,

    I have provided the images in my previous two posts. Please have a look and also please answer my doubts regarding your suggested workarounds. I am copying those again here:

    What is the use of passing RespectExifOrientation to decoder->GetPixelDataAsync then?

    I thought passing this flag will return me the rotated bitmap, am I wrong? How will I restore the original orientation in my destination image then?

    As illustration, say I have a image of pixels 640x480 so that its oriented dimensions are 480x640.

    So passing RespectExifOrientation to decoder->GetPixelDataAsync should give me the bitmap of 480x640 respecting the exif orientation. And if so, I passed the same values to encoder. What is the discrepancy?

    Monday, September 24, 2012 11:02 AM
  • Ok. I can reproduce this now with your picture. It looks like there may be a problem with the math when combining scaling and EXIF rotation, but you can avoid that by doing it yourself:

    • Check the orientation from imageFile->Properties->GetImagePropertiesAsync 
    • Set the transform's rotation based on the ImageProperties' Orientation.
    • Rotate the encoding height and width based on the new orientation
    • Set the ExifOrientationMode to IgnoreExifOrientation

    --Rob

    Friday, September 28, 2012 2:39 AM
    Owner
  • So, when you know it is a bug, are you going to fix it before final release?
    Tuesday, October 09, 2012 6:49 AM