none
compare depth-color image RRS feed

  • Question

  • hello people, I want to compare the data streams rendered by depth and color camera in C# and return them together in an images (sort of point cloud.. like depthwithcolor c++ sample), here is my code. it opens the window but then it crush.. any help?? 

             private void SensorAllFramesReady(object sender, AllFramesReadyEventArgs e)
            {
                using (DepthImageFrame depthFrame = e.OpenDepthImageFrame())
                using (ColorImageFrame colorFrame = e.OpenColorImageFrame())
                {
                    OutputImage = new WriteableBitmap(depthFrame.Width, depthFrame.Height, 96, 96, PixelFormats.Bgr32, null); 
                    
                    byte[] colorPixels = new byte[colorFrame.PixelDataLength];
                    short[] depthPixels = new short[depthFrame.PixelDataLength];
                    byte[] output = new byte[sensor.DepthStream.FramePixelDataLength * sizeof(int)];
                    int outputIndex = 0; 

                    if (depthFrame != null && colorFrame != null)
                    {
                        depthFrame.CopyPixelDataTo(depthPixels);
                        colorFrame.CopyPixelDataTo(colorPixels);

                        for (int y = 0; y < depthFrame.Height; y++)
                        {
                            for (int x = 0; x < depthFrame.Width; x++)
                            {
                                var depthIndex = x + (y * depthFrame.Width);
                                var playerIndex = depthPixels[depthIndex] & DepthImageFrame.PlayerIndexBitmask;
                                var colorPoint = sensor.MapDepthToColorImagePoint(depthFrame.Format, x, y, depthPixels[depthIndex], colorFrame.Format);
                                var colorPixelIndex= (colorPoint.X*colorFrame.BytesPerPixel)+(colorPoint.Y*colorFrame.BytesPerPixel*colorFrame.Width);

                                output[outputIndex] = colorPixels[colorPixelIndex + 0];
                                output[outputIndex + 1] = colorPixels[colorPixelIndex + 1];
                                output[outputIndex + 2] = colorPixels[colorPixelIndex + 2];
                                output[outputIndex + 3] = playerIndex > 0 ? (byte)255 : (byte)0; 
                                ++outputIndex;
                            }
                        }

                        OutputImage.WritePixels(new Int32Rect(0, 0, depthFrame.Width, depthFrame.Height),output,depthFrame.Width * depthFrame.BytesPerPixel,0);

                        Image.Source = OutputImage;

                    }
                }
                        
            } 

    Thursday, July 26, 2012 8:45 AM

All replies

  • With what exception (and on which line) does it crash?

    One bug I see (though I don't think it would be causing a crash), is that ++outputIndex should probably be outputIndex += 4 instead.

    I'd also recommend not calling MapDepthToColorImagePoint so many times... this is likely to be very slow. Instead, you should consider calling MapDepthFrameToColorFrame just once per frame.

    John
    K4W Dev

    Monday, July 30, 2012 11:26 PM
  • The issue could be that MapDepthToColorImagePoint can return out of range X,Y coordinates for depth values that don't have a value color mapping.

    If you look at the GreenScreen-WPF sample in the toolkit you'll notice there is a check for out of bounds data:

    // make sure the depth pixel maps to a valid point in color space
    // check y > 0 and y < depthHeight to make sure we don't write outside of the array
    // check x > 0 instead of >= 0 since to fill gaps we set opaque current pixel plus the one to the left
    // because of how the sensor works it is more correct to do it this way than to set to the right
    if (colorInDepthX > 0 && colorInDepthX < this.depthWidth && colorInDepthY >= 0 && colorInDepthY < this.depthHeight)
    { ... }
    

    I'm just guessing though...  As John mentioned, an exception description would help a lot.

    -- Jon

    Tuesday, July 31, 2012 5:59 AM