How i managed to do some gestures... Post your ideas here... RRS feed

  • General discussion

  • Hello all Kinect SDK enthousiasts!

    Ok i started this topic because i have managed to do some some gestures and i would like to see your comments on how i did it... or maybe help someone who haven't figured out a way yet...

    Note that i am 99% sure that there is a better way to recognize gestures

    Swipe to the left with right hand

    First i assign two public integers


    public int gesture1Helper = 0;
    public int gesture1Speed = 10; // The smaller the number the faster the speed of the gesture required to perform



    And then inside the nui_SkeletonFrameReady event (basically in every frame)




         if (getBodySegment(data.Joints, brush, JointID.HandRight).Points.First().X > 
          getBodySegment(data.Joints, brush, JointID.HipCenter).Points.First().X + 40)
          gesture1Helper = skeletonFrame.FrameNumber;
         if (getBodySegment(data.Joints, brush, JointID.HandRight).Points.First().X <
          getBodySegment(data.Joints, brush, JointID.HipCenter).Points.First().X - 40 &
          skeletonFrame.FrameNumber - gesture1Speed < gesture1Helper)
           gesture1Helper = skeletonFrame.FrameNumber - 20;
           //swipe to the left with right hand performed


    This is basically the swipe... any ideas on this concept are appreciated. :)


    Wednesday, June 22, 2011 10:14 AM

All replies

  • Thomas thanks for your code, was looking for something to begin and it has worked perfectly.
    Wednesday, June 22, 2011 10:27 PM
  • 1.

    Could you explain what it means in detail ?

    skeletonFrame.FrameNumber - gesture1Speed < gesture1Helper


     what do your application will implement via your gesture recognition ?


    Thank you!!!

    Wednesday, June 29, 2011 9:23 AM
  • I managed to get nice results by using the method I describe in the awnsers to this thread :
    Wednesday, June 29, 2011 10:08 AM
  • Yes of course.... i will try :)

    1. Basically i have set a minimum frame number (gesture1speed) that is required for the player to perform the gesture.

    That means in my sample code that the gesture must happen in less than 10 frames to be recognized.

    So lets take an example:

    a. My "right hand's X coordinate" is bigger than my "hip center's X coordinate + 40" that means that my right hand is right to my hip plus 40 pixels. So the current framenumber is saved in my gesture1helper variable.

    b. when my right hand X leaves the "hipcenter.X + 40" zone i don't track it and the variable gesture1helper holds the last frame number that my hand was in that hipcenter.X +40 zone. [lets say for example it happens that my hand leaves at the frame 100]

    c. then i track if my hand is at the "hipcenter.X - 40" zone... if it does i check the current frame... lets say in this example it is the frame 103.

    that basically means that my right hand moved from the "hipcenter.X + 40" to the "hipcenter.X - 40" in 3 frames.

    I hope that makes sense...

    2. My application at this point is a photo viewer...  just for fun! i load JPEG images from a folder and i can move back and forth with gestures


    Wednesday, June 29, 2011 7:08 PM
  • I also found a way to do the pinch (zoom in/out motion) movement with both hands...

    again i have some public variables....

        //My code for hand raising recognition
        int rightHandRaised = 0;
        int leftHandRaised = 0;
        //My code for pinch to the image
        float currentDistance = 0; //0 for no hand is raised
        double smooth = 0.1; //smoothness of move
        int changeAtEveryPinch = 20; // the pixels the image resizes every time<br/>



    Then inside the nui_SkeletonFrameReady i have this code for recognizing at first if both hands are raised..


    if (data.Joints[JointID.ShoulderRight].Position.Z - data.Joints[JointID.HandRight].Position.Z > 0.5)
                //My right hand is raised
                rightHandRaised = 1;
              else rightHandRaised = 0;
              if (data.Joints[JointID.ShoulderLeft].Position.Z - data.Joints[JointID.HandLeft].Position.Z > 0.5)
                //My Left hand is raised
                leftHandRaised = 1;
              else leftHandRaised = 0;



    And later in the same event if both hands are raised i track the distance between them to pinch... (zoom in/out motion)

    if (rightHandRaised == 1 && leftHandRaised == 1 && currentDistance==0)
                currentDistance = data.Joints[JointID.HandRight].Position.X - data.Joints[JointID.HandLeft].Position.X;
              if (rightHandRaised == 1 && leftHandRaised == 1)
                pinchImage(data.Joints[JointID.HandRight].Position.X, data.Joints[JointID.HandLeft].Position.X);
              if (rightHandRaised == 0 && leftHandRaised == 0) currentDistance = 0;


    Ok and here i have my crappy pinchImage function... plz dont point and laugh at this... :p

    void pinchImage(float rightX, float leftX)
          // debug2.Text = (rightX - leftX).ToString();
          if (rightX - leftX + smooth < currentDistance)
            if (image1.Width > changeAtEveryPinch && image1.Height > changeAtEveryPinch)
              image1.Height -= changeAtEveryPinch;
              image1.Width -= changeAtEveryPinch;
              currentDistance = rightX - leftX;
          if (rightX - leftX - smooth > currentDistance)
            image1.Height += changeAtEveryPinch;
            image1.Width += changeAtEveryPinch;
            currentDistance = rightX - leftX;

    Hope it helps anyone! Cheers!

    Wednesday, June 29, 2011 7:18 PM
  • Thanks for posting your code, Thomas. Newbies like me find it really useful.



    Wednesday, June 29, 2011 7:48 PM
  • Thank you for your patient explanation, I've tried and it performed excellently.
    Friday, July 1, 2011 2:52 AM
  • Thanks for posting your code, Thomas. It is worthful and broadens my mind.

    But I have two problems.

    1.Why use "0.5" and what it means ? It is whether proper or not if changes to another number, such as 1.0, 1.5 etc. ?

    data.Joints[JointID.ShoulderRight].Position.Z - data.Joints[JointID.HandRight].Position.Z > 0.5

    2.How to use the variable smooth ?


    Thank you ! ^_^


    Friday, July 1, 2011 3:32 AM
  • Thank you for you kind words Doraemon6061,

    Well to answer your question
    1. this number (0,5) should be a public variable but it is simply the depth distance between shoulder and hand. If the depth distance is above 0.5 it suppose that your hand is raised.

    2. smooth variable was because in my program i dont want to resize my image all the time (at every frame) so i used this variable to resize the picture less times on a gesture... sorry my explanation is not so good as English is not my native language.

    Saturday, July 2, 2011 5:23 PM
  • I've posted a couple of code samples here that might help!

    Sunday, July 3, 2011 1:40 AM
  • Thank you David this toolkit is really awesome !
    Tuesday, July 12, 2011 9:16 AM
  • Words cannot describe how grateful I am for your code example Thomas!  I was struggling, trying to get the toolkit posted above to work with my specific needs but couldn't get it to do so, but 2 seconds with modifying your example a tiny bit proved to be exactly what I needed! :D
    Wednesday, July 13, 2011 5:04 PM
  • Hi aswath87,

    How or where to get skeletonHistory.Add ?

    Or at least, how skeletonHistory should look ?


    Regards, Anatoly



    Wednesday, July 20, 2011 1:46 PM
  • Hey Anatoly, I just use a list for skeletonHistory to which I keep adding new SkeletonData objects. 

    List<SkeletonData> skeletonHistory. 

    Monday, July 25, 2011 4:49 PM
  • Can u please explain what gestures you are using for zooming in and out in this code?

    Sunday, February 9, 2014 7:38 AM