none
Is it possible to get the Infrared ColorImageFrame into a Texture2D? RRS feed

  • Question

  • Hi, I'm having trouble trying to put the byte[] of a frame into a Texture2D. Specifically the Infrared one (the others work fine). I am writing with XNA and wanted to add an infrared option, I don't want to start from scratch with WPF. The error I get is an unhandled exception "The size of the data passed in is too large or too small for this resource." I use

    The byte array is [614400] and the Texture2D is [307200], but
    the RGB version is [1228800] and the Texture2D is [307200] so it's not that.

    I was wondering if anyone had done this yet or if the formats of Texture2D mean it's not possible?? I tried to find away of using the WriteableBitmap from the WPF Toolkit example but to no avail.

    The code in question:

    ColorImageFrame colourVideoFrame = kinect.ColorStream.OpenNextFrame(1);
    
    //...
    
    if (colourVideoFrame != null)
    {
          Byte[] pixelData = new Byte[colourVideoFrame.PixelDataLength];
          colourVideoFrame.CopyPixelDataTo(pixelData);
    
          Texture2D temp = new Texture2D(graphicsDevice, colourVideoFrame.Width, colourVideoFrame.Height);
          temp.SetData(pixelData);
    }
    Saturday, November 10, 2012 8:32 PM

Answers

  • Even though the size of the color frame to represent the data is 640 x 480 x 4 = 1228800, the data in the colorFrame when using IR or Color streams will be different.

    // Turn on the color stream to receive color frames
    this.sensor.ColorStream.Enable(ColorImageFormat.InfraredResolution640x480Fps30);
    // Allocate space to put the pixels we'll receive
    this.colorPixels = new byte[this.sensor.ColorStream.FramePixelDataLength];

    InfraredResolution640x480Fps30 = 614400
    RgbResolution640x480Fps30 = 307200

    The data in the colorFrame will show how many bytes which is passed to the SetData function.

    IR colorFrame
    - colorFrame {Microsoft.Kinect.ColorImageFrame} Microsoft.Kinect.ColorImageFrame - base {Microsoft.Kinect.ColorImageFrame} Microsoft.Kinect.ImageFrame {Microsoft.Kinect.ColorImageFrame} BytesPerPixel 2 int FrameNumber 106554 int Height 480 int PixelDataLength 614400 int Timestamp 524515360 long Width 640 int RGB colorFrame
    - colorFrame {Microsoft.Kinect.ColorImageFrame} Microsoft.Kinect.ColorImageFrame - base {Microsoft.Kinect.ColorImageFrame} Microsoft.Kinect.ImageFrame {Microsoft.Kinect.ColorImageFrame} BytesPerPixel 4 int FrameNumber 103741 int Height 480 int PixelDataLength 1228800 int Timestamp 524421499 long Width 640 int

    Take into account every IR pixel is using 2 bytes and only the 10 most significant bits are used. This is similar to how the depth data mapped. You will have to pull that value and populate and RGB buffer to use with your Texture2D



    Tuesday, November 13, 2012 7:53 PM

All replies

  • Even though the size of the color frame to represent the data is 640 x 480 x 4 = 1228800, the data in the colorFrame when using IR or Color streams will be different.

    // Turn on the color stream to receive color frames
    this.sensor.ColorStream.Enable(ColorImageFormat.InfraredResolution640x480Fps30);
    // Allocate space to put the pixels we'll receive
    this.colorPixels = new byte[this.sensor.ColorStream.FramePixelDataLength];

    InfraredResolution640x480Fps30 = 614400
    RgbResolution640x480Fps30 = 307200

    The data in the colorFrame will show how many bytes which is passed to the SetData function.

    IR colorFrame
    - colorFrame {Microsoft.Kinect.ColorImageFrame} Microsoft.Kinect.ColorImageFrame - base {Microsoft.Kinect.ColorImageFrame} Microsoft.Kinect.ImageFrame {Microsoft.Kinect.ColorImageFrame} BytesPerPixel 2 int FrameNumber 106554 int Height 480 int PixelDataLength 614400 int Timestamp 524515360 long Width 640 int RGB colorFrame
    - colorFrame {Microsoft.Kinect.ColorImageFrame} Microsoft.Kinect.ColorImageFrame - base {Microsoft.Kinect.ColorImageFrame} Microsoft.Kinect.ImageFrame {Microsoft.Kinect.ColorImageFrame} BytesPerPixel 4 int FrameNumber 103741 int Height 480 int PixelDataLength 1228800 int Timestamp 524421499 long Width 640 int

    Take into account every IR pixel is using 2 bytes and only the 10 most significant bits are used. This is similar to how the depth data mapped. You will have to pull that value and populate and RGB buffer to use with your Texture2D



    Tuesday, November 13, 2012 7:53 PM
  • Thankyou, I used depth as a template and got something on the screen, although it's riddled with noise. So I must be doing something wrong because there is no noise in the WPF tutorial, also my intensity is wrong as the image is far brighter than it should be.

    private byte[] ConvertInfraFrame(byte[] pixelData, ColorImageStream infraStream) { byte[] infraFrame = new byte[infraStream.FrameWidth * infraStream.FrameHeight * 4]; //experiment: for (int i = 0, j = 0; j < pixelData.Length; i+=2, ++j ) { byte intensity = (byte)(~(pixelData[j]) >> 1); infraFrame[i] = intensity; infraFrame[i + 1] = intensity; } return infraFrame; }

    Which gets me this: i.imgur.com/ilHCu.jpg

    Apologies for comming back again, I just can't think of anything new to change - I was wondering if FrameNumber had something to do with it as it's the only different data not accounted for?

    Tuesday, November 20, 2012 1:20 AM
  • Take a look at the InfraredBasics samples, where they are just shifting the value to give you the 8 MSB.  If you want something exact but not efficient, then divide the value by the maximum 0xffc0 and calculate the RGB values. Be sure you are using 255 for the forth byte for an RGBA resource.

    Wednesday, November 28, 2012 10:27 PM