locked
How to pass CanvasPixelArray to C++ WinRT component for pixel level processing

    Question

  • HTML5 Canvas provide getImageData()/putImageData() API to access pixel array in JS, but how to pass it crossing WinRT ABI to underlying C++ component for faster processing?
    Tuesday, December 06, 2011 2:32 AM

Answers

  • After review, this will work:

     

         var canvasImage = new Image();
                canvasImage.src = "images/storelogo.png";
                canvasImage.alt = "images/storelogo.png";
                canvasImage.addEventListener("load", function () {
                    var canvas = id("scenario3Canvas");
                    var context = canvas.getContext("2d");
    
                    // Determine what height and width are needed to preserve the aspect ratio
                    canvas.width = parseInt(
                        document.documentElement.clientWidth * .5);
    
                    canvas.height = canvas.width;
    
                    context.drawImage(this, 0, 0, canvas.width, canvas.height);
                    var originalPixelData = context.getImageData(0, 0, canvas.width, canvas.height);
                    var winrt = new MyComponentDll1.MyWinRTComponent();
                    var bytes = new Uint8Array(originalPixelData.data);
                    var res= winrt.dosomethingwiththebytes(bytes);

    -Jeff

     C++ code is taking

    array<unsigned __int8>^ vec

     


    Jeff Sanders (MSFT)
    Friday, January 27, 2012 5:38 PM
    Moderator

All replies

  • Hi Jerry,

    Here are some options...

     

    You can use the blob and randomAccessStream.

    For example:

    var canvas = document.createElement("canvas");

    var blob = canvas.msToBlob();

    var stream = blob.msRandomAccessStream;

    then pass the stream object to your third party WinRT component as an IRandomAccessStream. The stream would contain the image data from the canvas encoded as a PNG.

    Or

     

    You can directly pass the canvas pixel data to a C++ DLL as an array<uint8>. Here’s an examples:

     

    JS:

    var canvas = document.createElement("canvas");

    canvas.width = 200;

    canvas.height = 200;

    document.body.appendChild(canvas);

    var ctx = canvas.getContext('2d');

    ctx.fillStyle = "rgb(150,29,28)";

    ctx.fillRect(0, 0, 10, 10);

    var imgData = ctx.getImageData(0, 0, 100, 100);

    var pixelArray = imgData.data;

    console.log("Pixel array for 100x100 has length:" + pixelArray.length); // prints 40000

    var rtObj = new WinRTComponentDll1.WinRTComponent();

    var res = rtObj.getValueAt(pixelArray,0);

    console.log("C++ says value at 0 in pixelArray is: " + res); // prints 150

     

    C++:

    int WinRTComponent::GetValueAt(array<unsigned __int8>^ arr, int i) {

     return arr[i];

    }

    -Jeff


    Jeff Sanders (MSFT)
    Tuesday, December 06, 2011 6:49 PM
    Moderator
  • Hi Jeff,

    Thanks for your sample code. When the JS pixelArray passed to C++ WinRT method with type "array<unsigned __int>^", debugger give me an exception code "type mismatch". My Win8 build is 8102, not sure if it's related to WinRT version?

    Jerry

    Wednesday, December 07, 2011 1:33 AM
  • Jerry,

    Could you send me a simple repro of the problem so I can play with it some?  You can email me from my blog:  http://blogs.msdn.com/jpsanders.  Email author on the right side.

    -Jeff


    Jeff Sanders (MSFT)
    Thursday, December 08, 2011 9:42 PM
    Moderator
  • Jeff,

    Can you please post the fix for the type mismatch problem?  I'm also getting the same error.

     

    Thanks,

    George

    Friday, January 13, 2012 8:10 AM
  • Hi George,

    I never received the sample of the problem to investigate further so I was not able to investigate further.  Do you have a simple repro?

    -Jeff


    Jeff Sanders (MSFT)
    Friday, January 13, 2012 12:35 PM
    Moderator
  • After review, this will work:

     

         var canvasImage = new Image();
                canvasImage.src = "images/storelogo.png";
                canvasImage.alt = "images/storelogo.png";
                canvasImage.addEventListener("load", function () {
                    var canvas = id("scenario3Canvas");
                    var context = canvas.getContext("2d");
    
                    // Determine what height and width are needed to preserve the aspect ratio
                    canvas.width = parseInt(
                        document.documentElement.clientWidth * .5);
    
                    canvas.height = canvas.width;
    
                    context.drawImage(this, 0, 0, canvas.width, canvas.height);
                    var originalPixelData = context.getImageData(0, 0, canvas.width, canvas.height);
                    var winrt = new MyComponentDll1.MyWinRTComponent();
                    var bytes = new Uint8Array(originalPixelData.data);
                    var res= winrt.dosomethingwiththebytes(bytes);

    -Jeff

     C++ code is taking

    array<unsigned __int8>^ vec

     


    Jeff Sanders (MSFT)
    Friday, January 27, 2012 5:38 PM
    Moderator
  • I am trying to do exactly as mentioned in this thread but I am getting compiler errors while using array<unsigned __int8>^ vec in my WinRT component. where is this defined?
    • Proposed as answer by krishna0808 Saturday, June 30, 2012 7:27 PM
    • Unproposed as answer by krishna0808 Saturday, June 30, 2012 7:27 PM
    Saturday, June 30, 2012 7:10 PM
  • Ah, found that array<T>^ -> Platform:Array<T> now.

    Saturday, June 30, 2012 7:29 PM