none
How to store Z coordinates of a joint using the kinect sdk RRS feed

  • Question

  • Hello, 

    i'm using Kinect SDK V1.0 and i want to store the Z coordinates of the right Hand in aim to process it and recongnize if it's  a push gesture or  a pull gesture !

    i'm beginner in this work so i just took the skeleton tracking example and i added some instructions to store Z coordinates into an array (which i think it's a bad idea because i have to define a length and also i don't know if it updates value in every frame or only when there is gesture ) and i compared Z values to recongnize the hand gesture.

    this is a piece of the code:

    const int skeletonCount = 6;
            const int POSLEN = 500; 
            bool closing = false;
            float[] postab = new float[POSLEN] ;
            
               
           
            Skeleton[] allSkeletons = new Skeleton[skeletonCount];
    
      void NewSensor_AllFramesReady(object sender, AllFramesReadyEventArgs e)
            {
                if (closing)
                {
                    return;
                }
    
                //detecter la première squelette
                Skeleton first = GetFirstSkeleton(e);
              
                if (first == null)
                {
                    return;
                }
                        
                
                Joint j = first.Joints[JointType.HandRight];
     
                if (j.TrackingState == JointTrackingState.Tracked)
                {
                            
                    txtHR.Text = "Position Main Droite:";
                    cd.Text = ("X: " + j.Position.X + "\nY: " + j.Position.Y + "\nZ: " + j.Position.Z);
                    postab[0] = 0; 
                    for (int i = 1; i < POSLEN; i++)
                    {
                        postab[i] = j.Position.Z;
                        posz.Text = ("Z=" + j.Position.Z);
                     
                        if (postab[i] > postab[i-1])
                        {
                            gest.Text = "PULL";
    
                        }
                        else if (postab[i] < postab[i-1])
                        {
                            gest.Text = " PUSH";
                        }
                        
                        i++;
                    }
    
                    
                }

    I can run the code but the textBlock always displays pull gesture, my question is:  

    is there any other method to do that ?

    how could i use or access a  history of Z positions during the tracking time ??

    thanks for any help ^^





    • Edited by Pink192 Monday, July 30, 2012 12:44 PM
    Monday, July 30, 2012 12:41 PM

Answers

  • Hello John, 

    Thanks alot for your answer, i test it and it works fine =)

    • Marked as answer by Pink192 Tuesday, July 31, 2012 8:18 AM
    Tuesday, July 31, 2012 8:18 AM

All replies

  • Here's an explanation for the observed behavior in your code as written:

    On every frame, your code is setting postab[0] to zero, and then filling all of the elements postab[1] through postab[499] with the same value (the value of j.Position.Z).

    The first time through your loop, postab[1] is greater than postab[0], so it sets your message to "PULL". On all subsequent iterations, the values in your comparisons are equal (they're all j.Position.Z), so the message doesn't change.

    (Also note: you have an extra i++ at the bottom of your loop, which will cause it to skip half of the iterations.)

     

    You'll need to pursue a different approach: Add only one value to your array each time the AllFramesReady event occurs. To do this, you'll need to declare a field to store the index to be written, and increment it on each frame. You'll also need to have some logic for when you reach the end of the array; probably "wrap around" to the beginning of the array again.

    As for analyzing when a "pull" or "push" action has occurred, you'll probably want to do more than a simple less-than or greater-than comparison between two adjacent frames. Frames arrive only 33 ms apart, the granularity of movement is 1 mm, and there is likely to be a certain amount of noise in the position measurements. You will likely want to define a minimum distance of movement, and a maximum period of time during which that movement can occur, in order to qualify as a "pull" or "push" action. For example, if the position changes by at least 0.1 m in 15 frames or less, you might recognize an action. You may also want to introduce some kind of delay (e.g., 30 frames) after recognizing an action, so that you don't get multiple recognitions in a row, or spurious recognitions caused by just moving your hand back to a "neutral" position. The constants I've suggested here are only examples... you'll likely need to experiment and tweak these values to get the effect you desire.

    Here's a (very simplistic) approach that might work:

    const int FramesPerAction = 15;
    const float ActionThreshold = 0.1f; // distance in meters
    const int IdleFramesAfterAction = 30; // Note: This should be greater than FramesPerAction!
    
    float[] postab = new float[FramesPerAction];
    int currentFrame = 0;
    int idleFramesRemaining = IdleFramesAfterAction;
           
      void NewSensor_AllFramesReady(object sender, AllFramesReadyEventArgs e)
            {
                if (closing)
                {
                    return;
                }
    
                //detecter la première squelette
                Skeleton first = GetFirstSkeleton(e);
              
                if (first == null)
                {
                    // Can't see a skeleton: Return to idle state
    
                    idleFramesRemaining = IdleFramesAfterAction;
                    return;
                }
    
                Joint j = first.Joints[JointType.HandRight];
     
                if (j.TrackingState != JointTrackingState.Tracked)
                {
                    // Can't see the hand: Return to idle state
    
                    idleFramesRemaining = IdleFramesAfterAction;
                    return;
                }
                    
                // If we're not in the idle state, check for an action
    
                float currentPos = j.Position.Z;
    
                if (idleFramesRemaining == 0)
                {
                    float comparePos = postab[currentFrame];
                    if (currentPos - comparePos > ActionThreshold)
                    {
                        gest.Text = "PULL";
                        idleFramesRemaining = IdleFramesAfterAction + 1;
                    }
                    else if (comparePos - currentPos > ActionThreshold)
                    {
                        gest.Text = "PUSH";
                        idleFramesRemaining = IdleFramesAfterAction + 1;
                    }
                }
    
                // Store current position and advance current frame index (regardless of whether we're idle)
    
                postab[currentFrame] = currentPos;
                currentFrame = (currentFrame + 1) % postab.Length;
    
                // If we're in the idle state, decrement the idle counter
    
                if (idleFramesRemaining > 0)
                {
                    --idleFramesRemaining;
                }
            }
    

    Note that this code starts out initially in the idle state, which allows time for the table to fill up, before it starts trying to detect any actions by performing comparisons.

    Please also note that I haven't tried to compile, much less run, this code, but I hope it gives you some ideas on how to proceed.

    John
    K4W Dev

     

     

    Monday, July 30, 2012 7:42 PM
  • Hello John, 

    Thanks alot for your answer, i test it and it works fine =)

    • Marked as answer by Pink192 Tuesday, July 31, 2012 8:18 AM
    Tuesday, July 31, 2012 8:18 AM