none
GetColorPixelCoordinatesFromDepthPixel in SDK 1.0 RRS feed

  • Question

  • Hi,

    I was using a sample of code found on Channel9 :http://channel9.msdn.com/coding4fun/kinect/Display-Kinect-color-image-containing-only-players-aka-background-removal

    I try to change it in order to work with the SDK 1.0. I think that i have to use the "MapDepthFrameToColorFrame", but how could i use it in order to have the background removed ?

    Thank you for your answers !

    Friday, February 10, 2012 2:49 PM

Answers

  • Using MapDepthFrameToColorFrame is one way to do this and will have the best performance for mapping from the DepthImage to the ColorImage. 

    To use MapDepthFrameToColorFrame you will need to first create an array of ColorImagePoint that is the same size as the DepthImage data array. After calling MapDepthFrameToColorFrame your ColorImagePoint array will be filled in with the mappings from the depth image to the color image. 

    To get the point on the color image that corresponds to the point on the depth image, you index into the ColorImagePoint array with the same index that you used to get the pixel data from the DepthFrame's data array.

    For example:

    If you have an array of data from the DepthFrame called "depthData" and you're checking a particular point to see if it is a player called "depthIndex" and your ColorImagePoint array is called "ColorImagePointArray":

    You can check to see if the pixel is on a player: if(depthData[depthIndex] & 7 != 0)

    And then the color pixel's location is: ColorImagePointArray[depthIndex]

    Friday, February 10, 2012 4:35 PM

All replies

  • Using MapDepthFrameToColorFrame is one way to do this and will have the best performance for mapping from the DepthImage to the ColorImage. 

    To use MapDepthFrameToColorFrame you will need to first create an array of ColorImagePoint that is the same size as the DepthImage data array. After calling MapDepthFrameToColorFrame your ColorImagePoint array will be filled in with the mappings from the depth image to the color image. 

    To get the point on the color image that corresponds to the point on the depth image, you index into the ColorImagePoint array with the same index that you used to get the pixel data from the DepthFrame's data array.

    For example:

    If you have an array of data from the DepthFrame called "depthData" and you're checking a particular point to see if it is a player called "depthIndex" and your ColorImagePoint array is called "ColorImagePointArray":

    You can check to see if the pixel is on a player: if(depthData[depthIndex] & 7 != 0)

    And then the color pixel's location is: ColorImagePointArray[depthIndex]

    Friday, February 10, 2012 4:35 PM
  • bbarefield has a great reply.  I have a very simple example posted in this thread, if you want to see it in action: http://social.msdn.microsoft.com/Forums/en-US/kinectsdknuiapi/thread/c39bab30-a704-4de1-948d-307afd128dab

    -Adam Smith [MSFT]

    Saturday, February 11, 2012 4:08 AM
  • Thanks to both of you for this help.

    @Adam, I already saw this post, but now i understand it much better thanks to bbarefield explaination.

    But i still have a problem in my formula because here is what i get.

    And here is my AllFrameReady method:

            private void AllFrameReady(object sender, AllFramesReadyEventArgs e)
            {
                var imageFrame = e.OpenColorImageFrame();
                var depthFrame = e.OpenDepthImageFrame();
    
                if (imageFrame == null || depthFrame == null)
                    return;
    
                if (_depthPixel.Length != depthFrame.PixelDataLength)
                {
                    _depthPixel = new short[depthFrame.PixelDataLength];
                    _mappedDepthLocations = new ColorImagePoint[depthFrame.PixelDataLength];
                }
    
                if (_colorPixel.Length != imageFrame.PixelDataLength)
                {
                    _colorPixel = new byte[imageFrame.PixelDataLength];
                    _playerBitmap = new WriteableBitmap(imageFrame.Width, imageFrame.Height, 96, 96, PixelFormats.Bgra32,
                                                        null);
                    _playerBits = new byte[imageFrame.Width*imageFrame.Height*4];
                    DepthToImage.Source = _playerBitmap;
                }
                imageFrame.CopyPixelDataTo(_colorPixel);
                depthFrame.CopyPixelDataTo(_depthPixel);
    
    
                // invisible background
                for (int i = 0; i < _colorPixel.Length; i += 4)
                {
                    _playerBits[i + 3] = 0;
                    _playerBits[i + 2] = 0;
                    _playerBits[i + 1] = 0;
                    _playerBits[i] = 0;
                }
                    SensorChooser.Kinect.MapDepthFrameToColorFrame(depthFrame.Format, _depthPixel, imageFrame.Format,
                                    _mappedDepthLocations);
    
                for (int i = 0; i < _depthPixel.Length; i++)
                {
                    if ((_depthPixel[i] & 7) != 0)
                    {
                        var point = _mappedDepthLocations[i];
    
                        if ((point.X >= 0 && point.X < 640) && (point.Y >= 0 && point.Y < 480))
                        {
                            int index = (point.Y*640 + point.X)*4;
                            _playerBits[index] = (byte) (_colorPixel[index]);
                            _playerBits[index + 1] = (byte)(_colorPixel[index + 1]);
                            _playerBits[index + 2] = (byte)(_colorPixel[index + 2]);
                            _playerBits[index + 3] = 255;
                        }
                    }
                }
    
                _playerBitmap.WritePixels(new Int32Rect(0, 0, _playerBitmap.PixelWidth, _playerBitmap.PixelHeight),
                                          _playerBits, _playerBitmap.PixelWidth*sizeof (int), 0);
    
            }

    Could you tell me what is wrong in my formula ?

    Thank you for your help.

    Monday, February 13, 2012 2:06 PM
  • It looks like you must have additional logic somewhere else in your app that's tinting pixels white?  Perhaps you have two images somewhere?  I took your AllFramesReady and replaced my handler in my sample from http://social.msdn.microsoft.com/Forums/en-US/kinectsdknuiapi/thread/c39bab30-a704-4de1-948d-307afd128dab - (I had to rename _depthPixel -> _depthPixels, _playerBits->_bitmapBits, etc to fit my code) and lo and behold, I saw a picture of me! :) 

    Bonus: if you change your inner loop logic to this, you'll eliminate the small "gaps" in the picture:

        int index = (point.Y * 640 + point.X) * 4;
        _bitmapBits[index] = (byte)(_colorPixels[index]);
        _bitmapBits[index + 1] = (byte)(_colorPixels[index + 1]);
        _bitmapBits[index + 2] = (byte)(_colorPixels[index + 2]);
        _bitmapBits[index + 3] = 255;
    
        if (point.X < 640)
        {
            _bitmapBits[index + 4] = (byte) (_colorPixels[index]);
            _bitmapBits[index + 5] = (byte) (_colorPixels[index + 1]);
            _bitmapBits[index + 6] = (byte) (_colorPixels[index + 2]);
            _bitmapBits[index + 7] = 255;
        }


    -Adam Smith [MSFT]

    Monday, February 13, 2012 11:44 PM