Inconsistent Shoulder / Elbow / Wrist Bones orientation RRS feed

  • Question

  • Hi guys,

    I am having issues with arm bones orientations, sometimes, I get some inconsistent values. I mean for example, on the right arm, given a shoulder/elbow orientation, I sometimes get a elbow/wrist orientation that is 180 degrees flipped on the bone axis and that is not possible for a human skeleton. My guess was that Kinect was already dealing with this and making sure that could not happen. It seems that I was wrong.

    At the moment, I would like to control a rigged model with Kinect bones orientations and I managed to make this work but I still have these impossible relative orientations that the Kinect is giving me. To make things clear, this is not a jitter issue, the Kinect is providing me those values not for a couple of frames but it sticks to it.

    I made an example scene of the issue I am experiencing : in fact I am using Carmine's script in order to smooth the orientations, I modified it a bit to show the bones orientation with colored face cubes.

    There is also a Kinect record in this archive that you can use with Kinect Studio.

    So I would like to know if you guys already ran into this and found a way to have a nicely Kinect controlled rigged model in Unity.

    Thanks for any input on this :).


    Thursday, January 15, 2015 8:35 PM

All replies

  • What you are describing can happen based on how the orientation data is computed. You just need to filter the data to determine if that is something you want to apply or not. if you use Kinect Studio, you can turn on the orientation visualizer for the body 3D view to see the location of the bone normal.

    Carmine Sirignano - MSFT

    Friday, January 16, 2015 12:06 AM
  • Thanks for this input Carmine. Having the joints orientation in Kinect Studio makes me realize that my coordinate transform for my custom model is not good.

    Friday, January 16, 2015 5:09 PM
  • Sorry about that, maybe I was too quick on that answer.

    When turning the orientation visualizer on, I noticed that the shoulders normal are not changing when twisting the arms on bone axis. But when I get the raw Elbow joint orientation representing the shoulder/elbow bone orientation (right ?), I have rotation values on the bone axis. So what am I doing wrong or misunderstanding there ?

    Friday, January 16, 2015 7:14 PM
  • There shouldn't be any rotation in the shoulder joint, that would mean something was disconnected from shoulder-center to shoulder. Maybe in football that happens a lot, but not in a regular person.

    Always remember the y component follows the bone direction to its parent.

    Carmine Sirignano - MSFT

    Monday, January 19, 2015 5:32 PM
  • Yes I think I get it. But what would be the best way to control a 3D model with Kinect bones orientation then ? 

    What I am doing right now for example is applying the orientation I get on the elbow joint to the shoulder of my 3D model as body.JointOrientations[JointType.ElbowRight] is the orientation of the right shoulder/elbow bone, am I correct ?

    Monday, January 19, 2015 7:58 PM
  • I am encountering the same problem when applying the jointorientation to a mesh using Ogre3D. Until now I can't make it to work properly.. The overall arm pose is right, but there is stillt this odd twist... 


    Here is a video of what is happening. See the twist for the ulna.

    YouTube Video

    • Edited by Glethien Tuesday, January 20, 2015 1:18 PM
    Tuesday, January 20, 2015 12:37 PM
  • Glad to see I am not the only one :).

    As Carmine suggested, we could detect that there is a twist by comparing with the bones normal and not apply the rotation to the joint but what to do with it then, that's my next question. As Kinect is sometimes (often) providing those values, I can't just wait for Kinect to give me a consistent value, so do I need to correct it myself ? I think this would be great if the SDK could do it for us :).

    Tuesday, January 20, 2015 3:23 PM
  • I know this thread is 2 years old, but has anyone come up with a solution for this?

    I'm having the exact same issues and I can't seem to find a proper answer:

    Sunday, January 15, 2017 8:59 PM