Aberration/noise/jitter in hands position RRS feed

  • Question

  • Hello

    I am in the process of finishing an interface to control an environment with the hand in front of 5 feet by 3 feet screen.

    I want to use the hands to click, move...

    I tried a lot of different ways ti find a natural way to do the different operation, using different king of distance reference ... because evrey solution had noise, blind spot and so on. Then after so many failure I decided to visualize the position (with 3d curve), speed, acceleration over time ans discovered than my problems where coming from the data provided by the skeleton tracking.

    mainly the hand, the wrist, the elbow jump but to a point where in a lot of case it should not happen :

    e.g with my arm extended in front of the kinect (i am at 2 meters from the vertical of the kinect which is mounted 2m2 high) I move it from left to right and back, infact I am rotating around my should. And the curves I get are far from being an arc of a circle the radius (depth) in particular is really jumpy and strange in certain position, to a point where the distance from the hand to the shoulder vary way to much.

    if you observe the shoulder/elbow distance or the elbow/wrist distance it can vary and jump a lot. But it should be nearly a constant. This mean that these constraints are not used or not used in the right way by the tracking.

    Even simulating a mouse with 2 hands (one for position one for click ) is tedious because of theses tracking mistakes. 

    Any ways to improve this behavior ? 

    Sunday, June 3, 2012 8:53 PM

All replies

  • TheMaa,

    Have you tried enabling Transform Smoothing to even out the jitters?

    James Ashley - Presentation Layer Architect at Razorfish Emerging Experiences

    Monday, June 4, 2012 5:20 PM
  • Does this happen when one of the points is "in front" of another? (As viewed in the Kinect Explorer.) I've had trouble when (eg) a hand is moved in front of the face. When this happens, the hand and head point jump around in an effort to have different XY coordinates. Also, hand and shoulder, or hand and elbow.

    Monday, June 4, 2012 6:27 PM

    I am using:

    HRESULT  hr = _p_nui_instance->NuiTransformSmooth( &skel_frame, NULL );

    which should turn smoothing to his default parameters

    without is is worse. Smooth just smooth the problem but I still get 20 cm jumps. If I extract the maximum (tip of the end) and use the x and y location and the depth there I get nearly a good arc of a circle. So the problem is in the skeleton code. I am just doing a visualization with angles computed from the skeleton points to see if the defaults are only on position but still respect angles.

    It happen more when the hand is directed to the camera (no the hand is not too close).

    Skeleton viewer seem to behave the same bit the lack of drawing curve make it a bad option to observe this. I can post an image


    Yes it happen mainly when my right hand is in front of me or going to the left side. It look like what you say. I'm sure there is some kind of fucked up constraints in the skeleton.

    Monday, June 4, 2012 9:49 PM
  • it is even more strange

    sometime this strange behavior vanish ans it is nearly ok for a while (10/20 seconds)

    it look like if the tracking change state (one stable one less stable)

    Monday, June 4, 2012 10:42 PM
  • The more I extend my arm the more it happen (I am pretty tall too nearly 2m)

    Infact in what the human guideline call the physical interaction zone, if I remember, it is way better.

    if I keep my elbow down (below my shoulder) and move only forearm it getting close to a smooth curve most of the time when I move. as soon I extend my arm the hand start to get jumpy. 

    So it seem there is a problem with the technology outside of this zone (one more time if I use the depth info directly it is stable, so the problem is introduced by the skeleton logic). 

    Monday, June 4, 2012 11:36 PM