none
In VGB how do we specify the gesture is it set to be True in completion of Gesture Movement RRS feed

  • Question

  • 1. I am recorded Pointing and Push Gestures and Swipe Gestures and Importing XEF files within VGB, In the New project with wizard how do we specify to the Gesture type if static gesture or a loop gesture (I do not mean Discrete or Continuous gestures), what I mean is the Value of Gesture should be set to TRUE once a specific gesture completes it motion, for example when we are doing Pointing and Push Gestures than I Select the range starting from the position where the Pointing Hand starts to moving forward and Ends the Selection when hand is moving backward and passes behind the same position where it started, so I need Gesture to be recognized as this cycle is complete, not as pointing hand moves beyond certain point in Z Axes, so ho do I specify that?

    2. Also During the Project setup Wizard it just asks about if the Gesture depends on specific Hand State like Grip, Lasso or Open Hand, but never asks for which hand state it should be locked to.

    3. Another Question is about Swipe Gestures, would it be doing them as Discrete Gestures or Continuous Gestures and at what position i should actually set to True and what should be the Ending point of that gesture (To be set as False) 

    Sunday, November 2, 2014 6:53 PM

Answers

  • Hello Omar,

    1. VGB does not directly support this concept, as the gesture type that you are selecting in the wizard corresponds to the machine learning algorithm that will be used in training. However, you can do this in code (listen for when a gesture detection result turns from 'true' to 'false') and/or use a combination of discrete and continuous gestures to get the desired behavior. I would suggest using multiple discrete gestures (pointing moving forward, and pointing moving backward) and then use a continuous gesture to determine when the cycle is complete. For example, the continuous gesture might go from 0 to 1 when the 'moving forward' gesture occurs, and then from 1 to 0 when the 'moving backward' gesture occurs. When the 'moving forward gesture' is first detected, you would store the progress value (let's say that it is reported as 0.3), and then you would wait for that same progress value to be hit when the 'moving backward' gesture is detected, to know that the cycle is complete.

    2. If you choose to enable Hand State in the wizard, then it will become an additional input during training. If you are trying to train a gesture with a specific hand state (like punch, which has a closed hand state) then the correct hand state will be learned from the frames that you tag for training. For example, if the majority of frames tagged as 'true' have a closed hand state, then the ML will learn to recognize the gesture only when the closed hand state is detected. If this does not happen, then be sure to include negative examples in your training set. For example, if you don't want the gesture to be recognized with an 'open' hand state, then you should have an example in your training set that includes this data.

    3. You should start by tagging the Discrete version of the gesture, since you will need to use a Discrete gesture to give context to the Continuous gesture result (which always reports a progress value, even when no gesture is occurring). When tagging the Discrete gesture, try to include all frames where the swipe motion is occurring, but exclude any preparatory or recovery motions as this will confuse the trainer. Also, Continuous gestures are much easier to tag if you have already tagged the corresponding Discrete gestures because you can use the 'Generate Tags' option to do the bulk of the work for you.

    Finally, you might find the following training material useful (the second video covers Continuous gestures):

    Part 1:http://channel9.msdn.com/Blogs/k4wdev/Custom-Gestures-End-to-End-with-Kinect-and-Visual-Gesture-Builder

    Part 2:http://channel9.msdn.com/Blogs/k4wdev/Custom-Gestures-End-to-End-with-Kinect-and-Visual-Gesture-Builder-part-2-

    VGB Whitepaper: http://aka.ms/k4wv2vgb

    Thanks,

    ~Angela

    Monday, November 3, 2014 3:28 AM