none
which format are images retrieved from Kinect stored in an ArrayBuffer? RRS feed

  • Question

  • Hi, 

    I want to take the image from the kinect device and put it into a canvas. I know that in the SDK examples there is a very similar thing, in the colorframe example, but it use a "webgl" canvas, and I want to do it in 2D. At this point, I have an ArrayBuffer with all the data from the image loaded, but I don't know how to get that ArrayBuffer converted in something useful to me. I saw some topics which made the conversion like this: 

    var arrayBufferView = new Uint8Array(arrayBuffer);
    var blob = new Blob([arrayBufferView], { type: "image/bpm" });
    var urlCreator = window.URL || window.webkitURL;
    var imageUrl = urlCreator.createObjectURL(blob);
    
    var imageElement = new Image();
    imageElement.src = imageUrl;
                
    bodyContext.drawImage(imageElement, colorCanvas.width, colorCanvas.height);

    It seems to work, but I don't know which format should put. I tried the most common formats, like PNG, JPG, GIF... Non of those worked. Do you have any idea? 

    Tuesday, June 2, 2015 5:25 PM

All replies

  • The SDK samples use a 2D context with the WebGL canvas, that should fit your need.

    The data you get from the sensor varies depending on the frame type, but it does not map directly to an image format that you would use to load a static image (bmp, jpeg, png, etc)

    For example: the Color frame is an array of bytes 1920 * 1080 * 4 packed R,G,B,A (assuming you used the function CopyConvertedFrameDataToArray with the RGBA argument)  In ImageHelpers.js, you can see that we are creating a 2D texture with the format RGBA:

    glContext.texImage2D(
        glContext.TEXTURE_2D, 
        0, 
        glContext.RGBA, 
        width, 
        height, 
        0, 
        glContext.RGBA, 
        glContext.UNSIGNED_BYTE, 
        new Uint8Array(imageBuffer));

    Depth, IR, and BodyIndex frames are byte arrays of size 512 * 424.  You will notice that the html samples for these frame types use a winrt component to convert the frame data to an RGBA array to be rendered similarly to color.

    If you want to use the html5 canvas, you will probably have to set your pixels manually.  Once you get an RGBA array, you can use CreateImageData and set the image data values from this buffer:

    http://www.w3schools.com/tags/canvas_createimagedata.asp

    Unfortunately, this might be slow.

    Tuesday, June 2, 2015 7:10 PM
    Moderator
  • Thanks for the answer!

    I was trying that approach, and it seems to be the right way, but I couldn't make it work. Now I do something like this:

    colorFrame.copyConvertedFrameDataToBuffer(colorBuffer.buffer, kinect.ColorImageFormat.rgba);
    colorFrame.close();
    colorBuffer.transferToArrayBufferAsync()
       .done(function (arrayBuffer) {
          var imgData = bodyContext.createImageData(1920, 1080);
          var ubuf = new Uint8Array(arrayBuffer);
             
          for (var i = 0; i < ubuf.length; i += 4) {
             imgData[i] = ubuf[i];   //red
             imgData[i + 1] = ubuf[i + 1]; //green
             imgData[i + 2] = ubuf[i + 2]; //blue
             imgData[i + 3] = ubuf[i + 3]; //alpha
          }
    
          bodyContext.putImageData(imgData, 0, 0);
    
          colorBuffer.close();
       });

    Like I said, it doesn't work. I only see the canvas, but not the image, so I assume that I'm doing something wrong. Do you see something bad in the code?

    Furthermore, you are complete right saying that this is a slow method, do you know a better method or is this the best way?


    • Edited by IvanRT Thursday, June 4, 2015 12:34 PM
    Thursday, June 4, 2015 12:30 PM