none
Using Kinect v2 jointOrientations along with Three.js skinnedMesh RRS feed

  • Question

  • So far now here is what i have done :

    - built my 3d model in blender and created bones.

    - assigned vertices to each bone.

    - successfully exported the skinned mesh to json format , then imported it in three.js scene , tried animating bones manually and everything works fine.

    The kinect bit:

    my problem is that im using javascript api which has very low support ( but getting better as i see ) , after lot of digging i finally managed to find body joints , and body joint orientations.

    right now im doing my test on the bone head ( in my three.js is the bone that connects neck to center of head ) , so i startred doing the following :

    - get the neck ( joint type 2 ) orientation which is a quaternion .

    - normalize the quaternion using three.js method http://threejs.org/docs/#Reference/Math/Quaternion  

     .normalize()


    - create Euler Rotation from this normalized quaternion using three.js method http://threejs.org/docs/#Reference/Math/Euler

    .setFromQuaternion ( q, order )

    when i apply rotation using :

      skinnedMesh.skeleton.bones[1].rotation.set(eu.x,eu.y,eu.z,eu.order)
    where eu , is the resulted Euler rotation from above steps , i get unaccurate values , and axis seem flipped , anyone can help with this ?

    Monday, March 9, 2015 11:45 PM

All replies

  • When rigging the model, what direction is the Y axis? To use the orientation data from Kinect, you need to ensure the direction of the bone aligns with the Y component of the quaternion. See image here: https://social.msdn.microsoft.com/Forums/en-US/31c9aff6-7dab-433d-9af9-59942dfd3d69/kinect-v20-preview-sdk-jointorientation-vs-boneorientation?forum=kinectv2sdk

    The rotation you get at a particular joint is the rotation of the bone from its parent in absolute world space. If you want a local rotation you will need to calculate that (Quaternion.inverse(parent) * joint.Orienation).


    Carmine Sirignano - MSFT

    Tuesday, March 10, 2015 6:43 PM
  •  Thanks alot for your quick answer carmen , first of all there are no resource that indicates how i can access joint parent via Javascript  , as the object only contains type and position, so i guess there must be some reference to indicate joints and bones relation so i can do this manually , i.e create a manual hierarchy in my javascript code to relate joints to each other.

    what i want to clarify is that head bone im doing my tests on in on y-axis , also all results i get from kinect (of my own neck ) , always show a 'Y' value of 0 - 0.2 , never goes beyond this , tough  i rotate my head in all directions , but guess a you have stated this might be the relative rotation

    If you could please supply a reference to joints and their parents , so i can try your solution , i will really appreciate it , thanks in advance.

    Edit :

    I have attached a screenshot for my mesh with different quatrnions applied to it , to make things more clear :



    • Edited by prollygeek Wednesday, March 11, 2015 5:59 AM
    Wednesday, March 11, 2015 3:55 AM
  • Hi,

    I know it's an old thread, but how did you manage to solve this? I am wishing to do a similar thing with a skinned mesh in threejs and kinect v2 output, and don't know how to map the absolute kinect position with the local/relative bone positions.

    Thanks,
    J

    Monday, March 19, 2018 11:39 AM