Getting Depth of a point in Color Space. RRS feed

  • Question

  • How to get the depth of a point in Color Space the correct way?

    I am developing an automatic score annotation system for Table Tennis.

    The abstract working of the system is as follows.

    Step 1). Detect the ball using openCV color blob detection, which gives the center coordinates of the ball in color image.

    Step 2). Map the depth frame to color frame and obtain the depth map coordinate corresponding to the color coordinate of the center and also the depth of the Table.

    Step 3).  Check if the ball hit the table or fell on the ground or hit the wrong side of the table.

    Step 4). Repeat for every new frame.

    Now, the problem is that I am not getting any acceptable results, Some of which are

    • Even when the ball is lying still on the table, there is a considerable variation in the depth while it should be nearly constant.(Nearly constant because of the small variation in the center coordinates due to slight tracking errors). But when I test the depth in Kinect Studio recording the depth data seems to be accurate which means there is something that I am not doing correctly.
    • When the ball is moving pretty fast, the tracking is lost.(I don't know why but my webcam performs better than Kinect in this case).

    The code that I've written to obtain the depth of the center coordinate is 

    float CKinectSources::getColorPixelDepth(HRESULT c_mapperInit, cv::Mat depthMat, int x, int y)
    	float depth = -1;
    	HRESULT hr;
    	UINT16* depthBuffer;
    	UINT depthBufferSize;
    	DepthSpacePoint* depthPoints = NULL;
    	if (SUCCEEDED(c_mapperInit)) {
    		depthPoints = new DepthSpacePoint[1080 * 1920];
    		cv::Mat tempDepth = depthMat;
    		hr = _coordinate_mapper->MapColorFrameToDepthSpace(512 * 424,
    			reinterpret_cast<UINT16*>(, 1920 * 1080, depthPoints);
    		if (SUCCEEDED(hr)) {
    			DepthSpacePoint p = depthPoints[x + 1920 * y];
    			if (p.X != -std::numeric_limits<float>::infinity() && p.Y != -std::numeric_limits<float>::infinity()) {
    				int depthX = static_cast<int>(p.X + 0.5);
    				int depthY = static_cast<int>(p.Y + 0.5);
    				if (SUCCEEDED(hr)) {
    					if ( {
    						if (depthX <= 512 && depthX > 0 && depthY <= 424 && depthY > 0)
    							depth =<UINT16>(depthX, depthY);
    			if (depthPoints != NULL) {
    				delete[] depthPoints;
    				depthPoints = NULL;
    	return depth;

    Is the above way to obtain the depth correct or not?

    How does Microsoft give the depth of the coordinate where the cursor is at on the color image in Kinect Studio?

    and How to make the system more robust?

    • Edited by S_batra Monday, April 25, 2016 11:51 AM
    Monday, April 25, 2016 10:03 AM