The VGB extracts features for learning including MuscleForce, MuscleTorque and MusclePower.
- How are those features calculated? Is there a paper that describes the method? Typically in Biomechanics an inverse dynamics approach is used to calculate those values from the kinematics (Kinect's skeleton provides the joint kinematics) and kinetics (external forces acting on the system - where does Kinect gets those from?)
- Is there (or planned to be) a way to call the functions that calculate the features through the SDK? for example, if my application required muscle torque estimation as output? or any of the other features for that matter?
- The VGB classifier training outputs the 'Top 10 contributing weak classifiers'. Is there a way to get output of fValues for all weak classifiers?
- VGB seems to have a pre-set cap on number of weak classifiers (1000) to be used in the learing. Can this value be set / modified? how?
- What is the criteria for using or rejecting inferred joint as weak classifiers? I have noticed both appear in the 'Top 10 contributing weak classifiers'
- Changed type Carmine Si - MSFTMicrosoft employee, Owner Wednesday, January 21, 2015 7:48 PM
VGB, or for that matter the Kinect SDK, has no feature support for muscle force. VGB only works off joint positional information. Muscle force demos on the Xbox are not features of the SDK, but can be realized based on the higher fidelity of the new sensor data that is provided.
The SDK can only provide joint and orientation information about tracked users.
Carmine Sirignano - MSFT
- Edited by Carmine Si - MSFTMicrosoft employee, Owner Wednesday, January 21, 2015 7:50 PM
Thank you for the response, but it appears that you have misunderstood my questions completely. Below is extract from the output of the build stage of the VGB:
In relation to my questions 1 & 2:
Step 2 of 4: Generating a Pool of Weak Classifiers
Using Hands data: No
Using Skeleton data : Yes
1/38 - Feature: DiffPositionX (62)
2/38 - Feature: DiffPositionY (18)
3/38 - Feature: DiffPositionZ (93)
4/38 - Feature: Angles (573)
5/38 - Feature: TimeSpaceAngles (89)
6/38 - Feature: Speed (107)
7/38 - Feature: VelocityX (76)
8/38 - Feature: VelocityY (75)
9/38 - Feature: VelocityZ (89)
10/38 - Feature: AngleVelocity (81)
11/38 - Feature: AngleAcceleration (85)
12/38 - Feature: MuscleForceX (50)
13/38 - Feature: MuscleForceY (53)
14/38 - Feature: MuscleForceZ (65)
15/38 - Feature: MuscleTorqueX (155)
16/38 - Feature: MuscleTorqueY (47)
17/38 - Feature: MuscleTorqueZ (174)
18/38 - Feature: MusclePower (69)
In relation to my question 4 (under Build step 3):
Step 3 of 4: Training Strong Classifier
-Evaluating classifier data for each example skeleton frame...
-Running AdaBoost using 8 (of 8 available) hardware threads...
Num weak classifiers: 1000
In relation to my questions 3 & 5 (under Build step 4):
Top 10 contributing weak classifiers:
MuscleTorqueX( SpineMid ) using inferred joints, fValue < -0.400003, alpha = 1.197646
MuscleTorqueX( SpineShoulder ) using inferred joints, fValue < -0.300003, alpha = 0.659523
Angles( Head, ShoulderLeft, Neck ) using inferred joints, fValue < 38.000000, alpha = 0.487340
Angles( Head, ShoulderRight, Neck ) using inferred joints, fValue < 38.000000, alpha = 0.432138
Speed^2( FootRight ) rejecting inferred joints, fValue >= 0.400000, alpha = 0.424726
VelocityZ( AnkleLeft ) rejecting inferred joints, fValue >= 0.599997, alpha = 0.419117
MusclePower( FootLeft ) rejecting inferred joints, fValue >= 4.000000, alpha = 0.396936
MuscleTorqueZ( ShoulderRight ) rejecting inferred joints, fValue >= -1.400003, alpha = 0.381479
MuscleTorqueZ( ShoulderLeft ) using inferred joints, fValue < 0.499997, alpha = 0.365908
MuscleTorqueX( SpineMid ) rejecting inferred joints, fValue >= -0.400003, alpha = 0.340869
There have been no announcements or plans to support/change the machine learning algorithms or expose that publically. This is internal feature functionality which is typically not something we document as it is internal detail not required to use the feature.
You will find more information on VGB in the documentation: https://msdn.microsoft.com/en-us/library/dn785304.aspx including detail on the input parameters for the detection technologies:
Carmine Sirignano - MSFT