none
Joint Orientation? RRS feed

  • Question

  • Having played a bit with both OpenNI and now the Kinect For Windows SDK, I was wondering if there is anything like SkeletonJointOrientation in the Joints? I'm trying to turn a Kinect Skeleton into a Bone Matrix for a Skinned Model, and I can handle most of it, but now I can't seem to find the Orientation of the Joint. Is there a way to calculate this?
    Thursday, June 16, 2011 9:59 PM

Answers

All replies

  • I have the same problem. Where can i acces to the orientation of the joints?
    Friday, June 17, 2011 8:42 AM
  • I found this on the documentation:

     

    NUI Skeleton Tracking
    The NUI Skeleton API provides information about the location of up to two players standing in front of the Kinect sensor array, with detailed position and orientation information

     

    (page 20 of the programming guide)

     

    Please, anyone knows how to acces it? because in the structures that the SDK gives to you there is no orientation anywhere.

     

    thanks

    Friday, June 17, 2011 12:31 PM
  • The official answer is that these aren't included in the SDK
    Friday, June 17, 2011 4:38 PM
  • So then how do we go about calculating the orientation of each joint? Could we navigate the skeleton hierarchy, and calculate the vector value between two joints and call that the Orientation?
    Friday, June 17, 2011 5:20 PM
  • we can not call it orientation. what should we do with hands and feet? also, what we do with the neck? choose the rigth or left shoulder? I hope they release a new version with the orientation of the joints, if not, i don't know for what can be useful the Microsoft kinect sdk.
    Friday, June 17, 2011 8:22 PM
  •  

    Hi,

     

    I don't know if this fits your needs, but reading through several blog postings all over the web, I've written up the following code:

     

    double JointAngle(Joint Upper, Joint Middle, Joint Lower)
      {
       
       Vector3 LowLeft = new Vector3();
       LowLeft.X = Middle.Position.X - Lower.Position.X;
       LowLeft.Y = Middle.Position.Y - Lower.Position.Y;
       LowLeft.Z = Middle.Position.Z - Lower.Position.Z;
    
       Vector3 HighLeft = new Vector3();
       HighLeft.X = Middle.Position.X - Upper.Position.X;
       HighLeft.Y = Middle.Position.Y - Upper.Position.Y;
       HighLeft.Z = Middle.Position.Z - Upper.Position.Z;
    
       double Dot;
       Dot = 0;
    
       if ((HighLeft.Magnitude > 0) & (LowLeft.Magnitude > 0))
       {
    
        LowLeft.Normalize();
        HighLeft.Normalize();
    
        Dot = HighLeft.DotProduct(LowLeft);
        Dot = Math.Acos(Dot) * 180 / Math.PI;
    
        return Dot;
       }
       else { return -1; }
      }
    


    This function should return a single angle for such an joint like the elbow or the knee in degrees. You need to specify the position of the central joint (e.g. elbow), and the neighboring joints (e.g. wrist, shoulder). I haven't testet it yet for joints with higher degress of freedom. The suggestions for this code snippet came from Naem Barons post in the OpenNI Forum (http://openni-discussions.979934.n3.nabble.com/OpenNI-dev-Kinect-Joint-Angles-td2669062.html) and I used Vector3 class which is available here (http://www.codeproject.com/KB/recipes/VectorType.aspx).

    Hope this helps!


    Sunday, June 26, 2011 7:41 PM
  • thanks for the code, Tobias. But it is really confusing. So to calculate the angle for knee, we should use:
    JointAngle(Joint Hip, Joint Knee, Joint Foot)

    right?

    What about end points? Like head and hands?? They are not central points. What should be the last parameter??

    • Edited by sinand99 Wednesday, January 18, 2012 3:17 PM
    Wednesday, January 18, 2012 3:17 PM
  • It is not possible to deterine the joint orientation directly from positions only - you could find a vector between the two joints, but there is no information in two positions that would indicate how much rotation about that vector is needed.  So any twisting is not detectable in the joint information only.

    If you did some analysis of the color image too, then it might be possible, but with only the positions it is mathematically impossible to determine the orientations...

    Thursday, January 19, 2012 10:03 PM
  • 我也在做这方面的东西,我的实现方法是用Kinect SDK提供的20个点重新定义19根骨头,每根骨头的朝向用两个点来计算。
    具体实现如下:

    1、首先定义19根骨头

    enum StickBones
    {
    	STB_NECK = 0, 
    	STB_SHOULDER_LEFT,
    	STB_SHOULDERS_RIGHT,
    	STB_SPINAL, 
    	STB_WAIST, 
    	STB_HIP_LEFT, 
    	STB_HIP_RIGHT, 
    	STB_THIGH_LEFT, 
    	STB_THIGH_RIGHT, 
    	STB_CRUS_LEFT, 
    	STB_CRUS_RIGHT, 
    	STB_FEET_LEFT, 
    	STB_FEET_RIGHT, 
    	STB_ARM_LEFT, 
    	STB_ARM_RIGHT, 
    	STB_FOREARM_LEFT, 
    	STB_FOREARM_RIGHT, 
    	STB_HAND_LEFT, 
    	STB_HAND_RIGHT, 
    };

    2、定义一个Kinect SDK的Skeleton到自定义骨头的转换格式

    struct Skel2STB
    {
    	int stb, skel1, skel2; // skel1--->skel2
    };

    3、定义转换列表

    Skel2STB BoneTemple[] =
    {
    	{STB_NECK,				NUI_SKELETON_POSITION_SHOULDER_CENTER,	NUI_SKELETON_POSITION_HEAD},
    	{STB_SHOULDER_LEFT,		NUI_SKELETON_POSITION_SHOULDER_CENTER,	NUI_SKELETON_POSITION_SHOULDER_LEFT},
    	{STB_SHOULDERS_RIGHT,	NUI_SKELETON_POSITION_SHOULDER_CENTER,	NUI_SKELETON_POSITION_SHOULDER_RIGHT},
    	{STB_SPINAL,			NUI_SKELETON_POSITION_SHOULDER_CENTER,	NUI_SKELETON_POSITION_SPINE},
    	{STB_WAIST,				NUI_SKELETON_POSITION_HIP_CENTER,		NUI_SKELETON_POSITION_SPINE}, 
    	{STB_HIP_LEFT,			NUI_SKELETON_POSITION_HIP_CENTER,		NUI_SKELETON_POSITION_HIP_LEFT}, 
    	{STB_HIP_RIGHT,			NUI_SKELETON_POSITION_HIP_CENTER,		NUI_SKELETON_POSITION_HIP_RIGHT}, 
    	{STB_THIGH_LEFT,		NUI_SKELETON_POSITION_HIP_LEFT,			NUI_SKELETON_POSITION_KNEE_LEFT}, 
    	{STB_THIGH_RIGHT,		NUI_SKELETON_POSITION_HIP_RIGHT,		NUI_SKELETON_POSITION_KNEE_RIGHT}, 
    	{STB_CRUS_LEFT,			NUI_SKELETON_POSITION_KNEE_LEFT,		NUI_SKELETON_POSITION_ANKLE_LEFT}, 
    	{STB_CRUS_RIGHT,		NUI_SKELETON_POSITION_KNEE_RIGHT,		NUI_SKELETON_POSITION_ANKLE_RIGHT}, 
    	{STB_FEET_LEFT,			NUI_SKELETON_POSITION_ANKLE_LEFT,		NUI_SKELETON_POSITION_FOOT_LEFT}, 
    	{STB_FEET_RIGHT,		NUI_SKELETON_POSITION_ANKLE_RIGHT,		NUI_SKELETON_POSITION_FOOT_RIGHT}, 
    	{STB_ARM_LEFT,			NUI_SKELETON_POSITION_SHOULDER_LEFT,	NUI_SKELETON_POSITION_ELBOW_LEFT}, 
    	{STB_ARM_RIGHT,			NUI_SKELETON_POSITION_SHOULDER_RIGHT,	NUI_SKELETON_POSITION_ELBOW_RIGHT}, 
    	{STB_FOREARM_LEFT,		NUI_SKELETON_POSITION_ELBOW_LEFT,		NUI_SKELETON_POSITION_WRIST_LEFT}, 
    	{STB_FOREARM_RIGHT,		NUI_SKELETON_POSITION_ELBOW_RIGHT,		NUI_SKELETON_POSITION_WRIST_RIGHT}, 
    	{STB_HAND_LEFT,			NUI_SKELETON_POSITION_WRIST_LEFT,		NUI_SKELETON_POSITION_HAND_LEFT}, 
    	{STB_HAND_RIGHT,		NUI_SKELETON_POSITION_WRIST_RIGHT,		NUI_SKELETON_POSITION_HAND_RIGHT}, 
    };

    4、转换函数,我是用Ogre来做的渲染引擎,所以计算什么的格式都用的Ogre

    void CWorld::transformBone(const Ogre::String& modelBoneName, int skelJoint, Ogre::Vector3 vflip)
    {
    	Ogre::Skeleton* skel = m_pMainChar->GetFirstEntity()->getSkeleton();
    	if (!skel)
    		return;
    	Ogre::Bone* bone = skel->getBone(modelBoneName);
    	if (!bone)
    		return;
    	Ogre::Quaternion qI = bone->getInitialOrientation();
    	Vector4 vPos1 = m_arSkelPoints[BoneTemple[skelJoint].skel1];
    	Vector4 vPos2 = m_arSkelPoints[BoneTemple[skelJoint].skel2];
    	Ogre::Vector3 vop2(vPos2.x, vPos2.y, vPos2.z);
    	Ogre::Vector3 vop1(vPos1.x, vPos1.y, vPos1.z);
    	vop1 *= vflip;
    	vop2 *= vflip;
    	Ogre::Vector3 vP = vop2 - vop1;
    	vP.normalise();
    	Ogre::Vector3 vd =  qI * -Ogre::Vector3::UNIT_Y;
    	Ogre::Quaternion newQ = vd.getRotationTo(vP);
    	bone->resetOrientation(); 
    	newQ = bone->convertWorldToLocalOrientation(newQ);
    	bone->setOrientation(newQ * qI);
    	
    }

    5、使用方法

    void CWorld::updateBone(Real deltaTime)
    {
    	NUI_SKELETON_FRAME SkeletonFrame = {0};
    	if ( SUCCEEDED(m_pNuiSensor->NuiSkeletonGetNextFrame( 0, &SkeletonFrame )) )
    	{
    		HRESULT hr = m_pNuiSensor->NuiTransformSmooth(&SkeletonFrame,NULL);
    		if(FAILED(hr))
    		{
    			return;
    		}
    		BOOL bTracked = FALSE;
    		for ( int i = 0 ; i < NUI_SKELETON_COUNT ; i++ )
    		{
    			if( SkeletonFrame.SkeletonData[i].eTrackingState == NUI_SKELETON_TRACKED)
    			{
    				for (int j = 0; j < NUI_SKELETON_POSITION_COUNT; ++j)
    				{
    					m_arSkelPoints[j] = SkeletonFrame.SkeletonData[i].SkeletonPositions[j];
    					bTracked = TRUE;
    				}
    				break;
    			}
    		}
    		Ogre::Vector3 vfp(1, -1, 1);
    		transformBone("Stomach",STB_SPINAL, Ogre::Vector3(1,1,-1));
    		transformBone("Waist", STB_WAIST, vfp);
    		transformBone("Root", STB_WAIST, vfp);
    		transformBone("Chest",STB_SPINAL, Ogre::Vector3(1,1,-1));
    		transformBone("Humerus.L",STB_ARM_LEFT, vfp);
    		transformBone("Humerus.R",STB_ARM_RIGHT, vfp);
    		transformBone("Ulna.L",STB_FOREARM_LEFT, vfp);
    		transformBone("Ulna.R",STB_FOREARM_RIGHT, vfp);
    		transformBone("Thigh.L",STB_THIGH_LEFT, vfp);
    		transformBone("Thigh.R",STB_THIGH_RIGHT, vfp);
    		transformBone("Calf.L",STB_CRUS_LEFT, vfp);
    		transformBone("Calf.R",STB_CRUS_RIGHT, vfp);
    	}
    }
    以上方法基本上可以实现对角色骨骼的控制了,不过没有加上多个角色的支持,只需要在updateBone函数中做些改动应该就可以了
    Tuesday, May 15, 2012 4:22 AM