Question about Joints, Input Streams and Output Stream... RRS feed

  • General discussion

  • Currently, I am trying to gather the position of the hand and move a mouse cursor with the hand. When I get the position of the hand, I scale it to this.Height / this.Width in order to scale it to the screen.  The issue is, a person only has a circular range of the hand, and I am trying to get it so it fits in to the rectangle.


    Any clue on how to scale the circular motion of the hand to the rectangular output?  

    Right now, I'm using this:          

          Microsoft.Research.Kinect.Nui.Vector LeftHandPosition = LeftHand.ScaleTo((int)this.Width, (int)this.Height).Position;


    Take a screen that is 1680x1050 for example... If the hand were at the position 420 , 262, the cursor should be at 0 , 0.. Additionally, 1260,262 -> 1680,0 ; 1260 , 788 -> 1680 , 1050 ; 420,788 -> 0,1050 etc etc



    Friday, October 7, 2011 8:29 PM

All replies

  • your strategy seems reasonable. Are you running into problems with it, or just wondering if there are better ways?


    I'm here to help
    Friday, October 7, 2011 11:40 PM
  • I'm running in to problems as I'm not quite sure how I can implement this to track the range of the arm appropriately. Perhaps there is some documentation or guides that I am missing? Currently, when a person reaches all the way to the right, it only goes to about half way between the right side and the center of the screen; directly up is the top of the screen, and to the waist only goes to the center of the screen (which is a problem as we don't want people to have to bend over to use this application.

    I hope this clarifies what my goal is better?



    • Edited by pros599 Tuesday, October 11, 2011 12:31 PM
    Tuesday, October 11, 2011 12:31 PM
  • Surely, I don't have to calibrate this per user by tracking both the right shoulder and hand, and then making the user reach out to the right (so I can get the length of the arm based on a 180 degree angle)...?


    I mean is there a better way than using the whole circle idea (ie: already implemented in the SDK)

    Tuesday, October 11, 2011 12:35 PM
  • No.

    However, it's pretty trivial to do this.  Here's some pseudo-code (since I don't know what language you're coding in).


    1. Initialize your camera.

    2. When the Skeleton is ready (i.e. in your skeleton-ready event handler), get a set of joints, in particular the shoulders, elbows, wrists and hands).

    3. Calculate the linear distance between each shoulder its respective hand joint.  As a result of this, do the simple math to determine the range of each arm.  It should be trivial for you to now have the geometry of the hand ranges.


    Tuesday, October 11, 2011 1:13 PM
  • That's exactly what I was afraid of... So Basically I'd have to have a calibration "phase" for each user?

    Also, I'm doing this in C# ... I guess the other huge issue I'm having is the math of assigning angle 0, radius X as the right most point on the horizontal (halfway between the top and the bottom of the screen).


    If that doesn't make a lot of sense, I could try to draw it out >.<

    Tuesday, October 11, 2011 1:19 PM
  • Well, yes you'd have to calibrate for each user.  However, it needn't be while they're stationary.  I guess I'm a little confused why this is frustrating for you since it's not as if you need to have a "Please wait while I calibrate phase".  It can be done in an instant.

    As for the math, I think you may be making it too complicated.  It's a simple algebraic equation: determine conceptually what angle will mark the perfect diagonal of your screen.  For example, let's use a 320x240 screen (to mirror the depth image).  We can determine the angle of that diagonal by asking what the arctan of 240/320 is, which is 0.6435 radians.  Now, we know the arm length L is going to be the hypotenuse of your selection-rectangle.  So, since we know the angle R and hypotenuse H of that rectangle, we can determine the other two sides.  sin(R) = O/H, so O (the height, i.e. Opposite side) = sin(R)*H.  cos(R) = A/H, so A (the width, i.e. Adjacent side) = cos(R)*H.

    Now you know the dimensions of the rectangle, and can easily plot any future hand position within that grid simply by tracking the Hand in the skeletonready event.

    Tuesday, October 11, 2011 5:20 PM
  • Hmm That makes sense.  

    I also found:  ScaleTo((int)this.Width, (int)this.Height, 0.5F, 0.4F).Position;  But, I'm not quite sure if that will be as good/accurate with the display and what not. Perhaps it would be easier and more accurate for me to do the calibration/user no?

    Tuesday, October 11, 2011 6:31 PM
  • Sounds like something you should test, and see which fits your needs best.
    Wednesday, October 12, 2011 4:58 PM