locked
byte array image rendering issue

    Question

  • hi Experts,

    Here is my scenario:

    I have a javascript UI layer and a C++/CX middle layer doing the background work.  During the working, middle layer will process a bitmap byte array and transfer to Javascript layer calling canvas.putImageData() to rendering from time to time.

    The rendering is frenquently happen and image is very big, so canvas.putImageData() is slow, which can not meet our requirement.

    I have some initial though, and hope you can direct me to the right way.

    Option 1 is try to conver the byte array to a image object(a bitmap object?)  which javascript layer can recognized, and call canvas.drawImage() to render(fromt the web, the data show drawImage is much faster than putImageData), the question is can I pass a bitmap object from C++ boundary to javascript? we do not use managed code, how to archieve this goal?

    Option 2 is try to pass canvas or canvas drawing buffer to C++, so the rendering happens on C++ side, still the question is can we pass a canvas object across boundary?

    Option 3 is convert byte array to image object in javascript layer, however I do not find a way to do this with out the canvas support..

    And if you have any other alternatives, please let me know! Thanks!!

    Wednesday, March 06, 2013 11:40 AM

Answers

  • Use Windows.Graphics.Imaging.BitmapEncoder to encode into a bitmap. This is the same in managed code and in native code.

    Alternately, you can use Windows Imaging Component (WIC). WIC is the same native code that System.Drawing.Image wraps.

    --Rob

    • Marked as answer by XXXing Tuesday, March 12, 2013 12:10 PM
    Friday, March 08, 2013 8:44 PM
    Owner
  • I know where I am wrong now.

    DataReader^ reader = ref new DataReader(randomAccessStream);
    should be changed as

    DataReader^ reader = ref new DataReader(randomAccessStream->GetInputStreamAt(0));


    • Marked as answer by XXXing Tuesday, March 12, 2013 12:10 PM
    Tuesday, March 12, 2013 12:10 PM

All replies

  • All of the drawing itself will need to be done by the HTML/JavaScript code. There is no way to inject images into the HTML UI from the C++ component.

    You should profile this to see where it is slow: is it slow to move the data from the component to the main app? Is it slow to load the decoded data? Is it a mix? etc. Without knowing where the performance problem is you will rely on guesswork to know where to optimize.

    Instead of passing along the raw bytes you can try encoding the bitmap into a jpg (or such) and then passing that along either in memory or via a temp file. Again, you will need to measure to see if it faster to encode, pass less data, and then decode or if it is faster to just pass the already decoded data.

    --Rob

    • Marked as answer by Jesse Jiang Monday, March 11, 2013 2:28 AM
    • Unmarked as answer by XXXing Monday, March 11, 2013 1:14 PM
    Wednesday, March 06, 2013 9:38 PM
    Owner
  • Thanks, Rob,

    From this link, http://jsperf.com/canvas-drawimage-vs-putimagedata/3 we can see the around 30 times performance gap between drawImage and putImageData..

    I am trying to do a encode the raw pixel to a png/jpeg image, then change it to base64 string and transfer to UI.

    if there any equivlent class in C++/Cx or win32 as in managed code like system.drawing.image does?

    ...

    MemoryStream ms = new MemoryStream(byteArrayIn);
    Image returnImage = Image.FromStream(ms);

    ...


    Friday, March 08, 2013 9:25 AM
  • Use Windows.Graphics.Imaging.BitmapEncoder to encode into a bitmap. This is the same in managed code and in native code.

    Alternately, you can use Windows Imaging Component (WIC). WIC is the same native code that System.Drawing.Image wraps.

    --Rob

    • Marked as answer by XXXing Tuesday, March 12, 2013 12:10 PM
    Friday, March 08, 2013 8:44 PM
    Owner
  • Thank you Rob!

    Now I am on the way to do the conversion, however, I am not good enough for the metro dev, I was stucked by the debugging... The string is not generated by my code, could you please have a review? Thank You very much!

    	task<String^> Update::GetPngBase64String()
    	{
    		auto randomAccessStream = ref new Windows::Storage::Streams::InMemoryRandomAccessStream();
    		DataReader^ reader = ref new DataReader(randomAccessStream);
    		return create_task(BitmapEncoder::CreateAsync(
    			BitmapEncoder::PngEncoderId, randomAccessStream)
    		).then([this](BitmapEncoder^ encoder){
    			encoder->SetPixelData(BitmapPixelFormat::Bgra8, 
    											BitmapAlphaMode::Straight,
    											this->width,
    											this->height, 
    											96,
    											96,
    											this->frameBuffer);
    			encoder->FlushAsync();
    		}).then([randomAccessStream, reader](){
    			reader->LoadAsync(randomAccessStream->Size);
    		}).then([reader, randomAccessStream](){
    			IBuffer^ buffer = reader->ReadBuffer(randomAccessStream->Size);
    			return "data:image/png;base64," + Windows::Security::Cryptography::CryptographicBuffer::EncodeToBase64String(buffer);
    		});
    	}

    Monday, March 11, 2013 1:23 PM
  • I know where I am wrong now.

    DataReader^ reader = ref new DataReader(randomAccessStream);
    should be changed as

    DataReader^ reader = ref new DataReader(randomAccessStream->GetInputStreamAt(0));


    • Marked as answer by XXXing Tuesday, March 12, 2013 12:10 PM
    Tuesday, March 12, 2013 12:10 PM