How to map 3D joint coordinates to 2D

• Question

• I've been able to find an algorithm to map a joint's x/y position to the screen, but I want to use the joint's Z position for Y.

Here's what I'm using:

``` public Vector3 GetJointCoordinates(Joint joint, double displayWidth, double displayHeight)
{
float depthX, depthY;
kinectSensor.SkeletonEngine.SkeletonToDepthImage(joint.Position, out depthX, out depthY);

// Convert the depth coordinates to 320x240 space.
const int depthImageWidth = 320;
const int depthImageHeight = 240;
depthX = Math.Max(0, Math.Min(depthX * depthImageWidth, depthImageWidth));
depthY = Math.Max(0, Math.Min(depthY * depthImageHeight, depthImageHeight));

// Get the video coordinates from the depth coordinates
int colorX, colorY;
ImageViewArea imageViewArea = new ImageViewArea();
kinectSensor.NuiCamera.GetColorPixelCoordinatesFromDepthPixel(ImageResolution.Resolution640x480, imageViewArea, (int)depthX, (int)depthY,
(short)0, out colorX, out colorY);

// The control displaying the image may not be 640x480, so scale the 640x480 coordinates to the width and height of the display.
const double videoWidth = 640.0;
const double videoHeight = 480.0;
return new Vector3((float)(displayWidth * colorX / videoWidth), (float)(displayHeight * colorY / videoHeight), 0);
}
```

Thursday, June 30, 2011 6:51 PM

• Shadowman1010, I think what you're after is what's known as "3D Projection." Try learning about that on Wikipedia or with a good computer graphics text. 3d projections can change both X and Y of the output pixel, not just the height.

On the other hand, if you just want to scale the depth values to the height of your image, then depth values vary from 800 to 4000, and you'd want to scale your output so that 800 is at the bottom, and 4000 at the top, or vice versa. Output pixel height as a percentage of output image height would be (depth-800)/(4000-800). But that won't give you a 3-D effect in the output image.
Friday, July 1, 2011 6:55 PM

All replies

I'm not sure what your question is here. What is it that you see when you run this code vs what you would like to see?

Only thing I can think of is, have you tried getting a depthValue by using SkeletonToDepthImage method that has one additional parameter, and then passing depthValue to GetColorPixelCoordinatesFromDepthPixel, instead of passing (short)0?

Hope this helps,
Eddy

I'm here to help
Friday, July 1, 2011 3:18 AM
• Eddy,

depthValue becomes an number around 2500.  I guess it's just a matter of subtracting some amount from it to map it to the window height.  That's what I'm having trouble with...the math; how to translate 3D depth onto 2D.

Friday, July 1, 2011 5:04 PM
• Shadowman1010, I think what you're after is what's known as "3D Projection." Try learning about that on Wikipedia or with a good computer graphics text. 3d projections can change both X and Y of the output pixel, not just the height.

On the other hand, if you just want to scale the depth values to the height of your image, then depth values vary from 800 to 4000, and you'd want to scale your output so that 800 is at the bottom, and 4000 at the top, or vice versa. Output pixel height as a percentage of output image height would be (depth-800)/(4000-800). But that won't give you a 3-D effect in the output image.
Friday, July 1, 2011 6:55 PM
• I don't need a 3D output.  It's basically moving a sprite along the Y axis as you move closer/farther from the kinect.  Thanks for the info on the depth values being 800-4000.  That helps!
Friday, July 1, 2011 9:38 PM