Answered by:
How to calculate the maximum and minimum of Joints in space
Question

Hello,
My question seems to be silly, but I bug for a while because I seek a more optimized method.
I want to search in the list of joint, the maximum and minimum coordinates over the 3 axes.
with these values (xmin, xmax, ymin, ymax, zmin and zmax) I want to calculate distances
W =  xmin  xmax ; H =  ymin  ymax ; D =  zmin  zmax 
and WD = sqrt (W * D * W + D)
at the same time i must calculate the system time whenever I find H and WD because it will help me to calculate celerity but that's not my concern at the moment ..
I tried several ways and methods:
example:public struct Vector3 { public float X; public float Y; public float Z; public Vector3(float x, float y, float z) { X = x; Y = y; Z = z; } public static float maxX(Vector3 left, Vector3 right) { float xmax ; if (left.X > right.X) xmax = left.X; else xmax = right.X; return xmax; } public static float maxY(Vector3 left, Vector3 right) { float ymax; if (left.Y > right.Y) ymax = left.Y; else ymax = right.Y; return ymax; } public static float maxZ(Vector3 left, Vector3 right) { float zmax; if (left.Z > right.Z) zmax = left.Z; else zmax = right.Z; return zmax; } public static float minX(Vector3 left, Vector3 right) { float xmin; if (left.X < right.X) xmin = left.X; else xmin = right.X; return xmin; } public static float minY(Vector3 left, Vector3 right) { float ymin; if (left.Y < right.Y) ymin = left.Y; else ymin = right.Y; return ymin; } public static float minZ(Vector3 left, Vector3 right) { float zmin; if (left.Z < right.Z) zmin = left.Z; else zmin = right.Z; return zmin; } public static float minimum(Vector3 left, Vector3 right, int C ) { float xmin, ymin, zmin; float rs = 0; switch (C) { case 1: { if (left.X < right.X) xmin = left.X; else xmin = right.X; rs = xmin; break; } case 2: { if (left.Y < right.Y) ymin = left.Y; else ymin = right.Y; rs = ymin; break; } case 3: { if (left.Z < right.Z) zmin = left.Z; else zmin = right.Z; rs = zmin; break; } } return rs; } public static float maximum(Vector3 left, Vector3 right, int C) { float xmax, ymax, zmax; float resultat = 0; switch (C) { case 1: { if (left.X > right.X) xmax = left.X; else xmax = right.X; resultat = xmax; break; } case 2: { if (left.Y > right.Y) ymax = left.Y; else ymax = right.Y; resultat = ymax; break; } case 3: { if (left.Z > right.Z) zmax = left.Z; else zmax = right.Z; resultat = zmax; break; } } return resultat; } public float HBox(Vector3 left, Vector3 right) { float H; float ymin = minimum(left, right, 2); float ymax = maximum(left, right, 2); H = Math.Abs(ymin  ymax); return H; } public float WDBox(Vector3 left, Vector3 right) { float W, D, WD; float xmin = minimum(left, right, 1); float xmax = maximum(left, right, 1); float zmin = minimum(left, right, 3); float zmax = maximum(left, right, 3); W = Math.Abs(xmin  xmax); D = Math.Abs(zmin  zmax); WD = (float)Math.Sqrt(Math.Pow(W, 2) + Math.Pow(D, 2)); return WD; } }
these methods allow me the comparison between two points but I wanted instead a method that makes me to loop the skeletal joints .. So i also tried
private void MinMaxDistanceFromCamera(Skeleton first) { foreach (Joint joint in first.Joints) { float jx = joint.Position.X; float jy = joint.Position.Y; float jz = joint.Position.Z; JlistX.Add(jx); JlistY.Add(jy); JlistZ.Add(jz); } foreach (float f in JlistX) { if (f < xMin) xMin = f; if (f > xMax) xMax = f; } foreach (float f in JlistY) { if (f < yMin) yMin = f; if (f > yMax) yMax = f; } foreach (float f in JlistZ) { if ((f < zMin) && (f != 0)) zMin = f; if ((f > zMax) && (f != 0)) zMax = f; } xmax.Text = xMax.ToString(); ymax.Text = yMax.ToString(); zmax.Text = zMax.ToString(); xmin.Text = xMin.ToString(); zmin.Text = zMin.ToString(); ymin.Text = yMin.ToString(); float W = System.Math.Abs(xMin  xMax); float H = System.Math.Abs(yMin  yMax); float D = System.Math.Abs(zMin  zMax); double WD = System.Math.Sqrt(Math.Pow(W, 2) + Math.Pow(D, 2)); }
With this last one, I can well calculate these values but after a while the values are blocking despite the image still displays ...
I want help to determine the value with the least calculation!
Do you have any idea please?
I'will be really glad for any help!
thank you a lot
DKF
Answers

You are on the right track with the second approach, but you can increase your time and space complexity by finding the min and max points in place:
// Initial values for min and max. // The bounding box will change as // the player moves, so these values // should be reset every frame. float minX = float.PositiveInfinity; float minY = float.PositiveInfinity; float minZ = float.PositiveInfinity; float maxX = float.NegativeInfinity; float maxY = float.NegativeInfinity; float maxZ = float.NegativeInfinity; foreach (Joint joint in first.Joints) { float jx = joint.Position.X; float jy = joint.Position.Y; float jz = joint.Position.Z; // Populate min and max X. if (jx < minX) { minX = jx; } if (jx > maxX) { maxX = jx; } // Populate min and max Y if (jy < minY) { minY = jy; } if (jy > maxY) { maxY = jy; } // Populate min and max Z if (jz < minZ) { minZ = jz; } if (jz > maxZ) { maxZ = jz; } }
In your example, you are iterating over your list a few times per frame, and you are also creating 3 additional lists for your X, Y, and Z values. Over time, you will start to use up more and more memory from these lists and your garbage collector will kick in and slow down your application.
Also, you are saving the text of the min and max coordinates. Make sure you are not printing these to your debug log every frame. That will also negatively impact your performance.
 Proposed as answer by Jackson Fields [MSFT]Microsoft employee Wednesday, May 27, 2015 3:39 PM
 Edited by Jackson Fields [MSFT]Microsoft employee Wednesday, May 27, 2015 3:41 PM updated code language for codebox
 Marked as answer by Jackson Fields [MSFT]Microsoft employee Friday, May 29, 2015 6:05 PM

The problem is that the joints are null. I am assuming that somewhere in your code you are setting the SkeletonStream's tracking mode to seated.
Please make sure your SkeletonStream's tracking mode is default (not seated)
A side effect of this is that your Trace and Plot functions are not handling the null case:
GetCoordinates is outputting some default value for x and y when it cannot find the desired joint in your joint collection.
Possibly change GetCoordinates to return false if the joint could not be found, then you can return early from the trace and plot functions before rendering.
 Marked as answer by pink192y Friday, May 29, 2015 8:38 AM
All replies

You are on the right track with the second approach, but you can increase your time and space complexity by finding the min and max points in place:
// Initial values for min and max. // The bounding box will change as // the player moves, so these values // should be reset every frame. float minX = float.PositiveInfinity; float minY = float.PositiveInfinity; float minZ = float.PositiveInfinity; float maxX = float.NegativeInfinity; float maxY = float.NegativeInfinity; float maxZ = float.NegativeInfinity; foreach (Joint joint in first.Joints) { float jx = joint.Position.X; float jy = joint.Position.Y; float jz = joint.Position.Z; // Populate min and max X. if (jx < minX) { minX = jx; } if (jx > maxX) { maxX = jx; } // Populate min and max Y if (jy < minY) { minY = jy; } if (jy > maxY) { maxY = jy; } // Populate min and max Z if (jz < minZ) { minZ = jz; } if (jz > maxZ) { maxZ = jz; } }
In your example, you are iterating over your list a few times per frame, and you are also creating 3 additional lists for your X, Y, and Z values. Over time, you will start to use up more and more memory from these lists and your garbage collector will kick in and slow down your application.
Also, you are saving the text of the min and max coordinates. Make sure you are not printing these to your debug log every frame. That will also negatively impact your performance.
 Proposed as answer by Jackson Fields [MSFT]Microsoft employee Wednesday, May 27, 2015 3:39 PM
 Edited by Jackson Fields [MSFT]Microsoft employee Wednesday, May 27, 2015 3:41 PM updated code language for codebox
 Marked as answer by Jackson Fields [MSFT]Microsoft employee Friday, May 29, 2015 6:05 PM

Hello and thanks for your answer !
Actually, in the second approach i used the field of view of Kinect :
float xMin = 2.2000f; float xMax = 2.2000f; float yMin = 1.6000f; float yMax = 1.6000f; float zMin = 4; float zMax = 0;
So i don't see the difference between what i did and what you propose except the no using of the two lists ?
does PositiveInfinity changes something compared to what i did ?
About calculating H and WD i was in need to display values so i can be sure that my program works fine !
is there another methode to save theese values out ?
thanks again ! :)
DKF

your initial min and max values look like they will work. PositiveInfinity and NegativeInfinity is just a way to guarantee that the algorithm will work for arbitrary spaces.
In terms of perf, you should see the most benefits from iterating over a single list without creating additional lists as I showed above.
If you're printing out the values to guarantee that your calculations are correct, make sure you remove or comment that out when you know the math is correct. Alternatively, you could put a breakpoint after the distance calculation and check that the values make sense in your debug windows.
 Edited by Jackson Fields [MSFT]Microsoft employee Wednesday, May 27, 2015 5:49 PM

Hello again !
i changed my function as follow :
public void box_dimensions(Skeleton sk) { foreach (Joint j in sk.Joints) { float jx = j.Position.X; float jy = j.Position.Y; float jz = j.Position.Z; // Populate min and max X. if (jx < minX) { minX = jx; } if (jx > maxX) { maxX = jx; } // Populate min and max Y if (jy < minY) { minY = jy; } if (jy > maxY) { maxY = jy; } // Populate min and max Z if (jz < minZ) { minZ = jz; } if (jz > maxZ) { maxZ = jz; } } float W = System.Math.Abs(minX  maxX); float H = System.Math.Abs(minY  maxY); float D = System.Math.Abs(minZ  maxZ); float WD = (float)Math.Sqrt(Math.Pow(W, 2) + Math.Pow(D, 2)); TextWriter tw = new StreamWriter("d:\\test.txt", true); tw.WriteLine(H); tw.WriteLine(WD); tw.Close(); }
in the SkeletonFramesready Event i did :
void newSensor_SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e) { using (SkeletonFrame skeletonFrame = e.OpenSkeletonFrame()) { if (skeletonFrame == null) return; skeletonFrame.GetSkeletonsT(ref skeletons); if (skeletons.All(s => s.TrackingState == SkeletonTrackingState.NotTracked)) { txtP.Text = "No Person detected"; return; } txtP.Text = "A person is detected"; skeletonDisplayManager.Draw(skeletons); foreach (Skeleton sk in skeletons) { skeletonDisplayManager.box_dimensions(sk); } } } }
the frame bugs i suppose because of the calculating rate, so i added a small condition which consists on manipulate 6 FPS and program runs with not bugs :
int framesCounter = 0; void newSensor_SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e) { framesCounter++; framesCounter = framesCounter % 5; // 5 is the divisor to work with 6 FPS if (framesCounter == 0) { using (SkeletonFrame skeletonFrame = e.OpenSkeletonFrame()) { if (skeletonFrame == null) return; skeletonFrame.GetSkeletonsT(ref skeletons); if (skeletons.All(s => s.TrackingState == SkeletonTrackingState.NotTracked)) { txtP.Text = "No Person detected"; return; } txtP.Text = "A person is detected"; skeletonDisplayManager.Draw(skeletons); foreach (Skeleton sk in skeletons) { skeletonDisplayManager.box_dimensions(sk); } } } }
Do you think that there is a good idea to create the box ?
is there any way to display that box once created ?
thanks again for your help !
DKF

A few more perf improvements you can make:
// Changing to max  min reduces an absolute value call. // However, you don't need to make these calculations, // because a bounding box only requires the min and max. //float W = maxX  minX; //float H = maxY  minY; //float D = maxZ  minZ; // You don't need this calculation either for the bounding box. //float WD = (float)Math.Sqrt(Math.Pow(W, 2) + Math.Pow(D, 2)); // If this is necessary, initialize it in your constructor. // Right now, this is allocating redundant memory inside a loop. //TextWriter tw = new StreamWriter("d:\\test.txt", true);
You'll note I changed the W, H, D calculations to be max  min. This will be slightly more performant since you can remove the absolute value call. However, you'll also note that I commented them out entirely. This is because a bounding box can be defined with just the min and max points with the following vertices:
min
(min.x, max.y, min.z)
(max.x, min.y, min.z)
(max.x, max.y, min.z)
(min.x, min.y, max.z)
(min.x, max.y, max.z)
(max.x, min.y, max.z)
maxLook at the rendering logic in skeletonDisplayManager.Draw. This function probably has some logic for rendering a skeleton bone (with a start position and end position) You can use the same rendering logic to draw the bounding box, connect the points above in the correct order to visualize the box.
If you make these changes, you should be able to get the app running at full frame rate rather than every 6th frame.
 Proposed as answer by Jackson Fields [MSFT]Microsoft employee Thursday, May 28, 2015 8:39 AM

thanks alot !
i did these changes, it's true that reduces code and i will make changes to draw the bounding box.
however the draw method i used in the skeletondisplayManager has a problem to draw the bottom part of the skeleton ! (i'm not using the seated mode so i delete it )
the bottom part is alway like this even when i walk correctly in the view filed of Kinect:
have you any idea where can the problem be ?
public void Draw(Skeleton[] skeletons, bool seated) { rootCanvas.Children.Clear(); foreach (Skeleton skeleton in skeletons) { if (skeleton.TrackingState != SkeletonTrackingState.Tracked) continue; Plot(JointType.HandLeft, skeleton.Joints); Trace(JointType.HandLeft, JointType.WristLeft, skeleton.Joints); Plot(JointType.WristLeft, skeleton.Joints); Trace(JointType.WristLeft, JointType.ElbowLeft, skeleton.Joints); Plot(JointType.ElbowLeft, skeleton.Joints); Trace(JointType.ElbowLeft, JointType.ShoulderLeft, skeleton.Joints); Plot(JointType.ShoulderLeft, skeleton.Joints); Trace(JointType.ShoulderLeft, JointType.ShoulderCenter, skeleton.Joints); Plot(JointType.ShoulderCenter, skeleton.Joints); Trace(JointType.ShoulderCenter, JointType.Head, skeleton.Joints); Plot(JointType.Head, JointType.ShoulderCenter, skeleton.Joints); Trace(JointType.ShoulderCenter, JointType.ShoulderRight, skeleton.Joints); Plot(JointType.ShoulderRight, skeleton.Joints); Trace(JointType.ShoulderRight, JointType.ElbowRight, skeleton.Joints); Plot(JointType.ElbowRight, skeleton.Joints); Trace(JointType.ElbowRight, JointType.WristRight, skeleton.Joints); Plot(JointType.WristRight, skeleton.Joints); Trace(JointType.WristRight, JointType.HandRight, skeleton.Joints); Plot(JointType.HandRight, skeleton.Joints); Trace(JointType.ShoulderCenter, JointType.Spine, skeleton.Joints); Plot(JointType.Spine, skeleton.Joints); Trace(JointType.Spine, JointType.HipCenter, skeleton.Joints); Plot(JointType.HipCenter, skeleton.Joints); Trace(JointType.HipCenter, JointType.HipLeft, skeleton.Joints); Plot(JointType.HipLeft, skeleton.Joints); Trace(JointType.HipLeft, JointType.KneeLeft, skeleton.Joints); Plot(JointType.KneeLeft, skeleton.Joints); Trace(JointType.KneeLeft, JointType.AnkleLeft, skeleton.Joints); Plot(JointType.AnkleLeft, skeleton.Joints); Trace(JointType.AnkleLeft, JointType.FootLeft, skeleton.Joints); Plot(JointType.FootLeft, skeleton.Joints); Trace(JointType.HipCenter, JointType.HipRight, skeleton.Joints); Plot(JointType.HipRight, skeleton.Joints); Trace(JointType.HipRight, JointType.KneeRight, skeleton.Joints); Plot(JointType.KneeRight, skeleton.Joints); Trace(JointType.KneeRight, JointType.AnkleRight, skeleton.Joints); Plot(JointType.AnkleRight, skeleton.Joints); Trace(JointType.AnkleRight, JointType.FootRight, skeleton.Joints); Plot(JointType.FootRight, skeleton.Joints); } }
DKF

Hard to tell from looking at the provided code, but here are some things I would look for:
Can you verify that your SkeletonStream's tracking mode is not set to seated?
You should not see the line kinect.SkeletonStream.TrackingMode = SkeletonTrackingMode.Seated anywhere in your code.
If you debug the code, can you verify that skeleton.Joints contains nonnull values for the joints that are not rendered?
If you do have values for the missing joints, do the values make sense? Do they have a smaller y value than the known joints above it (eg: the head)
What does the code for Trace and Plot look like?

Well i had to disturb my freind and let him walking ahead the Kinect ! :D
he was right in the view filed and i debug the program !
i can see that: Spine, Hip Center, Hip left, Hip Right, Knee Left, Knee right,Ankle left, Ankle right, Foot left and Foot right are not tracked and have null value ..
this is wired cause like at said, he was right in the view filed..
There are the Trace and plot methods !
readonly Canvas rootCanvas; readonly KinectSensor sensor; void Plot(JointType centerID, IEnumerable<Joint> joints) { float centerX; float centerY; GetCoordinates(centerID, joints, out centerX, out centerY); const double diameter = 8; Ellipse ellipse = new Ellipse { Width = diameter, Height = diameter, HorizontalAlignment = HorizontalAlignment.Left, VerticalAlignment = VerticalAlignment.Top, StrokeThickness = 4.0, Stroke = new SolidColorBrush(Colors.Green), StrokeLineJoin = PenLineJoin.Round }; Canvas.SetLeft(ellipse, centerX  ellipse.Width / 2); Canvas.SetTop(ellipse, centerY  ellipse.Height / 2); rootCanvas.Children.Add(ellipse); } void Plot(JointType centerID, JointType baseID, JointCollection joints) { float centerX; float centerY; GetCoordinates(centerID, joints, out centerX, out centerY); float baseX; float baseY; GetCoordinates(baseID, joints, out baseX, out baseY); double diameter = Math.Abs(baseY  centerY); Ellipse ellipse = new Ellipse { Width = diameter, Height = diameter, HorizontalAlignment = HorizontalAlignment.Left, VerticalAlignment = VerticalAlignment.Top, StrokeThickness = 4.0, Stroke = new SolidColorBrush(Colors.Green), StrokeLineJoin = PenLineJoin.Round }; Canvas.SetLeft(ellipse, centerX  ellipse.Width / 2); Canvas.SetTop(ellipse, centerY  ellipse.Height / 2); rootCanvas.Children.Add(ellipse); } void Trace(JointType sourceID, JointType destinationID, JointCollection joints) { float sourceX; float sourceY; GetCoordinates(sourceID, joints, out sourceX, out sourceY); float destinationX; float destinationY; GetCoordinates(destinationID, joints, out destinationX, out destinationY); Line line = new Line { X1 = sourceX, Y1 = sourceY, X2 = destinationX, Y2 = destinationY, HorizontalAlignment = HorizontalAlignment.Left, VerticalAlignment = VerticalAlignment.Top, StrokeThickness = 4.0, Stroke = new SolidColorBrush(Colors.Green), StrokeLineJoin = PenLineJoin.Round }; rootCanvas.Children.Add(line); }
DKF

The problem is that the joints are null. I am assuming that somewhere in your code you are setting the SkeletonStream's tracking mode to seated.
Please make sure your SkeletonStream's tracking mode is default (not seated)
A side effect of this is that your Trace and Plot functions are not handling the null case:
GetCoordinates is outputting some default value for x and y when it cannot find the desired joint in your joint collection.
Possibly change GetCoordinates to return false if the joint could not be found, then you can return early from the trace and plot functions before rendering.
 Marked as answer by pink192y Friday, May 29, 2015 8:38 AM
