none
VGB classifier features and learning parameters - details please

    General discussion

  • The VGB extracts features for learning including MuscleForce, MuscleTorque and MusclePower.

    1. How are those features calculated? Is there a paper that describes the method? Typically in Biomechanics an inverse dynamics approach is used to calculate those values from the kinematics (Kinect's skeleton provides the joint kinematics) and kinetics (external forces acting on the system - where does Kinect gets those from?)
    2. Is there (or planned to be) a way to call the functions that calculate the features through the SDK? for example, if my application required muscle torque estimation as output? or any of the other features for that matter?
    3. The VGB classifier training outputs the 'Top 10 contributing weak classifiers'. Is there a way to get output of fValues for all weak classifiers?
    4. VGB seems to have a pre-set cap on number of weak classifiers (1000) to be used in the learing. Can this value be set / modified? how?
    5. What is the criteria for using or rejecting inferred joint as weak classifiers? I have noticed both appear in the 'Top 10 contributing weak classifiers'

    Thanks.

    Wednesday, January 21, 2015 12:52 AM

All replies

  • VGB, or for that matter the Kinect SDK, has no feature support for muscle force. VGB only works off joint positional information. Muscle force demos on the Xbox are not features of the SDK, but can be realized based on the higher fidelity of the new sensor data that is provided.

    The SDK can only provide joint and orientation information about tracked users.


    Carmine Sirignano - MSFT


    Wednesday, January 21, 2015 7:48 PM
    Owner
  • Thank you for the response, but it appears that you have misunderstood my questions completely. Below is extract from the output of the build stage of the VGB:

    In relation to my questions 1 & 2:

    Step 2 of 4: Generating a Pool of Weak Classifiers
     Using Hands data: No
     Using Skeleton data : Yes
      1/38 - Feature: DiffPositionX (62)
      2/38 - Feature: DiffPositionY (18)
      3/38 - Feature: DiffPositionZ (93)
      4/38 - Feature: Angles (573)
      5/38 - Feature: TimeSpaceAngles (89)
      6/38 - Feature: Speed (107)
      7/38 - Feature: VelocityX (76)
      8/38 - Feature: VelocityY (75)
      9/38 - Feature: VelocityZ (89)
     10/38 - Feature: AngleVelocity (81)
     11/38 - Feature: AngleAcceleration (85)
     12/38 - Feature: MuscleForceX (50)
     13/38 - Feature: MuscleForceY (53)
     14/38 - Feature: MuscleForceZ (65)
     15/38 - Feature: MuscleTorqueX (155)
     16/38 - Feature: MuscleTorqueY (47)
     17/38 - Feature: MuscleTorqueZ (174)
     18/38 - Feature: MusclePower (69)

    ...

    In relation to my question 4 (under Build step 3):

    Step 3 of 4: Training Strong Classifier
     -Evaluating classifier data for each example skeleton frame...
    Done
     -Running AdaBoost using 8 (of 8 available) hardware threads...
    Done
     Num weak classifiers: 1000

    In relation to my questions 3 & 5 (under Build step 4):

    Top 10 contributing weak classifiers:
     MuscleTorqueX( SpineMid ) using inferred joints, fValue < -0.400003, alpha = 1.197646
     MuscleTorqueX( SpineShoulder ) using inferred joints, fValue < -0.300003, alpha = 0.659523
     Angles( Head, ShoulderLeft, Neck ) using inferred joints, fValue < 38.000000, alpha = 0.487340
     Angles( Head, ShoulderRight, Neck ) using inferred joints, fValue < 38.000000, alpha = 0.432138
     Speed^2( FootRight ) rejecting inferred joints, fValue >= 0.400000, alpha = 0.424726
     VelocityZ( AnkleLeft ) rejecting inferred joints, fValue >= 0.599997, alpha = 0.419117
     MusclePower( FootLeft ) rejecting inferred joints, fValue >= 4.000000, alpha = 0.396936
     MuscleTorqueZ( ShoulderRight ) rejecting inferred joints, fValue >= -1.400003, alpha = 0.381479
     MuscleTorqueZ( ShoulderLeft ) using inferred joints, fValue < 0.499997, alpha = 0.365908
     MuscleTorqueX( SpineMid ) rejecting inferred joints, fValue >= -0.400003, alpha = 0.340869

    Thanks

    Wednesday, January 21, 2015 8:52 PM
  • There have been no announcements or plans to support/change the machine learning algorithms or expose that publically. This is internal feature functionality which is typically not something we document as it is internal detail not required to use the feature.

    You will find more information on VGB in the documentation: https://msdn.microsoft.com/en-us/library/dn785304.aspx including detail on the input parameters for the detection technologies:

    https://msdn.microsoft.com/en-us/library/dn785523.aspx


    Carmine Sirignano - MSFT

    Friday, January 23, 2015 7:10 PM
    Owner