none
Kinect sensor mapping functions running slow ?? RRS feed

  • Question

  • Im using Kinect in project to use it in gesture recognition, hand & finger tracking.

    Problem is that in the new SDK when use for ex. Kinectsensor.MapDepthFrameToColorFrame() it takes long time & decrease el frame rate strongly.


    In this project we need to segment the hands from the scene, so when all frames are ready i take the depth frame & make processing on it to only show the hands by thresholding.

    Then i take the processed thresholded depth frame and map it to the RGB frame using Kinectsensor.MapDepthFrameToColorFrame() here is the code of mapping:

     

    byte[] colorFrame = new byte[width * height * 4]; // 32-bit valid Image

    int positionIn32BitImg;

    ColorImagePoint[] colorPoint = new ColorImagePoint[width * height];

    private byte[] GenerateColoredBytes(short[] depthData, int width, int height)

            {            

                var playerIdx = 0;

                var distance = 0;

     

                if (handRX != 0)

                {                

                    

    // Kinect sensor object

                    sensor.MapDepthFrameToColorFrame(DepthImageFormat.Resolution640x480Fps30, depthData,       ColorImageFormat.RgbResolution640x480Fps30, colorPoint);                ************ This function takes too long time ************

                    

                    for (int i16 = 0; i16 < width * height; i16++)

                    {

                        distance = GetDistanceFromCam(depthData[i16]);

                        playerIdx = GetPlayerIndex(depthData[i16]);

     

                        positionIn32BitImg = (colorPoint[i16].Y * width + colorPoint[i16].X) * 4;

                        if ((distance >= (handZ * 1000) - 70) && (distance <= (handZ * 1000) + 70) && playerIdx != 0) // in mm

                        {

                            colorFrame[positionIn32BitImg + BLUE_IDX] = 255;

                            colorFrame[positionIn32BitImg + GREEN_IDX] = 255;

                            colorFrame[positionIn32BitImg + RED_IDX] = 255;                        

                        }

                        else

                        {

                            colorFrame[positionIn32BitImg + BLUE_IDX] = 0;

                            colorFrame[positionIn32BitImg + GREEN_IDX] = 0;

                            colorFrame[positionIn32BitImg + RED_IDX] = 0;

                        }

     

                    }

                }

                return colorFrame;

            }

     

    Anyone knows why??

    Wednesday, March 7, 2012 7:59 PM

Answers

  • It's fast enough that you should be able to run in real-time, but we have noted that it could definitely be faster.

    For now, I'd recommend incrementing an index rather than doing multiplication to find it each pixel.

    In general addition is much faster than multiplication, and tightening up that loop will improve your fps.

    Also make sure you're only doing the mapping once per Kinect depth frame.

    Friday, March 9, 2012 6:16 PM