none
How to set Bodies to hand tracking?

    Question

  • Kinect can get up to 6 bodies, but I think that hand tracking is only available for two bodies. Or does all bodies hand tracking active?

    In Kinect v1 there is SkeletonStream.ChooseSkeletons() method to set skeletons to track. Is there an equivalent to choose bodies to hand tracking?

    Monday, July 21, 2014 8:15 AM

Answers

All replies

  • Yes, when you have a tracked body you want to have handStates for, call this.kinectSensor.BodyFrameSource.OverrideHandTracking(body.TrackingId);

    Carmine Sirignano - MSFT

    Monday, July 21, 2014 6:52 PM
    Owner
  • Thank you Carmine, I will check this.
    Tuesday, July 22, 2014 7:27 AM
  • So, if I want to choose 2 certain bodies out of the tracked 6, I just call OverrideHandTracking() 2 times with the appropriate tracking ids?


    marher - dx

    Saturday, November 08, 2014 5:46 PM
  • Yes, for the 2 people you want hand states, just override with the ids you want.

    Technically, there is a way to get hand states for all bodies, by always cycling the tracking ids. This would not be at 30fps, but depending on the scenario, it may be something you wish to try as well.


    Carmine Sirignano - MSFT

    Monday, November 10, 2014 11:52 PM
    Owner
  • Does that mean, that when Kinect Handler receives a new frame, the closest 2 bodies already have handstates - which took an amount of calculation time, and when I now want to get handstates for my tracked 2 different bodies via OverrideHandTracking(), it slows the system down, as it has to calculate 2 more handstates?

    marher - dx

    Tuesday, November 11, 2014 11:32 AM
  • Does that mean, that when Kinect Handler receives a new frame, the closest 2 bodies already have handstates - which took an amount of calculation time, and when I now want to get handstates for my tracked 2 different bodies via OverrideHandTracking(), it slows the system down, as it has to calculate 2 more handstates?

    marher - dx

    I don't think so, although I haven't tried it. When you set the body IDs with OverrideHandTracking, all body frames arriving after should have the hand states for these bodies without extra calculations. Carmine, that's how I think it is?
    Tuesday, November 11, 2014 11:54 AM
  • There is no assumptions that can be made about which ones have hand states. If you are tracking 6 people, only 2 will have active hand tracking states the others will just return no tracked state.

    Carmine Sirignano - MSFT

    Tuesday, November 11, 2014 7:13 PM
    Owner
  • But I assume its not a random generator that selects the bodies for handstates, so it would be very helpful to know the general algorithm for handstate selection - btw. I would have thought it would be always the 2 closest bodies... Can you please shed some more light on this, thanks.

    marher - dx

    Wednesday, November 12, 2014 7:22 AM
  • Use the API to set the tracking ID, that is the only way to know which body will have tracking id.

    As with body tracking, there is no way to know which body is the person being tracked. This is an active system that continually checks for possible tracking candidates until it locks onto a body. You can never assume a index value is going to be a particular user.


    Carmine Sirignano - MSFT

    Wednesday, November 12, 2014 9:03 PM
    Owner
  • Ok, so is there a way to turn the unpredictable default handstate tracking off, and just use OverrideHandTracking() for my selected bodies? And if not, I would like to suggest this as a little SDK improvement. Thanks,

    marher - dx


    • Edited by marher Thursday, November 13, 2014 7:31 AM
    Thursday, November 13, 2014 7:01 AM
  • If you have not used the OverrideHandTracking, then you would assume that it is in a unknown state. In your application this can be a global flag. If it is set, just ignore any hand states until you have set the tracking bodies.

    Carmine Sirignano - MSFT

    Thursday, November 13, 2014 6:08 PM
    Owner
  • Is there any performance drawback, if I set my favourite body candidates for HandState-tracking every frame or is iit just enough to do this once, and Handstates are locked onto the body as long as it is visible to the Kinect?

    marher - dx

    Friday, November 14, 2014 7:38 AM
  • Just do it once. Handstates will only be calculated for the body tracking id's you calculated. Keep in mind this is a per frame state of the hand, and can flicker. It would be up to you to add filtering to stabilize the state of the hand.

    https://social.msdn.microsoft.com/Forums/en-US/4ebc1cb6-694f-45d4-9e8a-36d63c8feec1/bodyhandleftstate-handstate-false-positives?forum=kinectv2sdk


    Carmine Sirignano - MSFT

    Friday, November 14, 2014 7:03 PM
    Owner
  • Hi, if I have told Kinect to track 2 favourite bodies via OverrideHandTracking(), how can I remove HandState-tracking from these bodies again?

    Regards,


    marher - dx

    Wednesday, November 26, 2014 7:40 AM
  • You can set the value to 0, but as a best practice, I would just ignore hand states based on a flag I set from within the application instead.

    Carmine Sirignano - MSFT

    Wednesday, November 26, 2014 6:42 PM
    Owner
  • Thanks Carmine for the suggestion. What would you do - as best practice - in the scenario, where a lot of people are moving around the Kinect and 2 of them should gain control over the Kinect application. I have used a "login gesture" with handstates. When they are done, they execute a "logout gesture" - with different handstates and 2 other guys should seamlessly (without requesting the audience to move outside Kinects influence sphere) be able to login and proceed. So, if nobody is logged onto the system, the users, who are closest to the centre are assigned with handstates

    var chosenones = bodies.Where(b=>b.IsTracked)

    .OrderBy(b=>b.Joints[Head].Position.X)

    .ThenBy(b=>b.Joints[Head].Z)

    .Take(2);

    So my scenario is more about shifting / assigning handstates than turning handstates on and off...


    marher - dx

    Monday, December 01, 2014 10:04 AM
  • I would never arbitrarily pick people based on location. I would take closest as a possible candidate but only through a gesture (lifting arm in the "PHiZ" region) would it consider them for actively engaging. See our "Human Interface Guidelines (HIG)" since that covers most scenarios.


    Carmine Sirignano - MSFT

    Monday, December 01, 2014 8:53 PM
    Owner
  • What if in the scenario it cannot be assumed, that the crowd in front of the Kinect is cooperative, which means, that everyone wants to engage immediately?

    PHiZ is fine, but if 2 people try to engage simultaneously, for me it seems more natural, that the guy standing just in front of Kinect (i.e. in the centre) should gain control, not the moron at the corner, that tries to spoil my demo :)


    marher - dx

    Tuesday, December 02, 2014 8:45 AM
  • Look at the controls basics sample to employ the a strategy that works. In that sample, when someone is engage whether to the side or in-front, no one else can take over control.

    You can also employ a round-robin system where each body gets so many frames to calculate hand states over the 30fps. This is similar to how we did multiple user background removal in v1. Since v1 only supported 2 tracked users, but we know about 6, we can alternate between all of them and give each some time to calculate out the information needed:

    http://blogs.msdn.com/b/kinectforwindows/archive/2013/10/22/using-kinect-background-removal-with-multiple-users.aspx

    With hand states since this can fluctuate, you need to do more filtering until you lock-on to 2 users.


    Carmine Sirignano - MSFT

    Tuesday, December 02, 2014 7:09 PM
    Owner