Skeleton and hand gesture recognition at the same time possible? RRS feed

  • General discussion

  • Hello,

    i am writing a thesis for my bachelor degree about an audio only videogame at the moment. The Interaction should work with the Kinect gesture recognition.

    Now i need to know if it is possible to track the sceleton movement (sitting, only arms and head movement) and to recognize gestures with the hand at the same time with one Kinect.

    If that was the case, i could use the arms for actions like fighting or defending and the handgestures for movement and other operations (i absolutly need the Headtracking for virtual surround).

    It is a theoretical work and i know how the Kinect recognition works, but i dont have any development skills and i couldnt find any references so far, i would be thankfull for any help.

    pardon my english, best regards, Hensol

    Tuesday, December 9, 2014 3:20 PM

All replies

  • VGB tech is based on body tracking information. To train gestures that the application will be looking for, you would have to be able to capture the body joint locations of the system. If you can capture the body data using Kinect Studio, then you will be able to train on that data

    As for using the Gesture in your application, you can get both body data and detect the gesture at the same time. This is a sample that is provided.

    As for head orientation, there is no tracking or ability to use that in VGB at this time.

    Carmine Sirignano - MSFT

    Tuesday, December 9, 2014 7:14 PM
  • Thank you for answering my questions, that information is very helpful.

    I have another question about the head orientation. I know there are quite precise applications for tracking head poses with the Kinect RGBD camera (with 3D face masks) and now i know it is possible to detect gestures and body movement at the same time.

    Is ist possible to detect all three informations at the same time (theoreticly, regarding only the opportunities of the Kinect without considering the needed applikations thet may not have been programmed yet)?

    Best regards, Hensol

    Friday, December 12, 2014 11:09 AM
  • We provide head orientation information from the Face api's. See Face tracking basics sample in the SDK browser. This will not work in VGB since KStudio recordings only record data that comes from the core runtime only.

    Carmine Sirignano - MSFT

    Friday, December 12, 2014 8:13 PM
  • Ok, but regardless of the applications that do allready exist, is it theoretical possible for a game developer to use the Kinect as a tool for interaction in the way that the devise detects head orientation, hand gestures and skeleton data at the same time?

    As i mentioned before, it is just for the theoretical konception of a game. pardon my lacking dev skills, the answer to the question might be obvious for professionals, but it would be kind if you help me again.

    Best regards, Hensol

    Friday, December 12, 2014 11:38 PM
  • sure the body joint and orientation data come from the core Kinect SDK where head orientation come from additional tech called the face API's. You might want to have a look at the Jumpstart series of videos (model 1,2,3 and 5) for more in-depth coverage of the tech and what is available.

    Carmine Sirignano - MSFT

    Monday, December 15, 2014 7:13 PM
  • Thank you, good to know that it works!
    Monday, December 15, 2014 9:38 PM
  • I watched the videos as recommended and the possibilities for my game are just amazing. The head orientation tracking may work perfectly to adjust the head in the game seperate from the body and make the localisation of soundsources way more accurate. Also now i do know  how to pronounce your name right ;)

    It might be a little much to ask but it would really valuraze my work if i could get a quote from you to support my research. Something that summerizes my questions, like "yes, it is possible to use the kinect as an input devise for your application, it is possible to get the data for the head orentation, the body data and the gesture data at the same time" or something like that.

    even without the quote you have been a great help, thank you Mr. Sirignano.

    Best regards, Hensol

    Tuesday, December 16, 2014 1:07 PM
  • No problem, that is why we are here. It is a service of the Kinect for Windows team to have an engineer provide support in the forums to assist the community. If it is more of a critical on-going engagement you need, please reach out to our MVP or partners:

    Carmine Sirignano - MSFT

    Tuesday, December 16, 2014 7:09 PM