none
Joint Orientation Smoothing [Unity C#]

    General discussion

  • Hi there!

    We all know that Kinect's raw orientation data has a lot of noise, but with a Smooth Filter you can erase that noise just adding a bit of lag. I have made a simple script which is providing good results (Attach it to a cube):

    using UnityEngine;
    using System.Collections;
    using System.Collections.Generic;
    using Windows.Kinect;
    
    public class JointRotationSmoothController : MonoBehaviour
    {
    	public JointType jointType;
    	public BodySourceManager bodyManager;
    	public int queueSize = 15;	
    	private Queue<Quaternion> rotations;
    	
    	void Start () 
    	{
    		this.rotations = new Queue<Quaternion>();
    		for (int i = 1; i < this.queueSize; i += 1)
    		{
    			this.rotations.Enqueue(this.transform.rotation);
    		}
    	}
    
    	void Update ()
    	{
    		foreach (KeyValuePair<ulong, Body> pairIdBody in this.bodyManager.GetBodies())
    		{
    			if (this.bodyManager.OrderOf(pairIdBody.Key) == 0)
    			{
    				ApplyJointRotation(pairIdBody.Value);
    			}
    		}
    	}
    	
    	private void ApplyJointRotation(Body body)
    	{
    		Quaternion lastRotation = Utils.GetQuaternion(body.JointOrientations[this.jointType]) * Quaternion.FromToRotation(Vector3.up, Vector3.forward);
    		this.rotations.Enqueue(lastRotation);
    		this.transform.rotation = SmoothFilter(this.rotations, this.transform.rotation);
    		this.rotations.Dequeue();
    	}
    	
    	private Quaternion SmoothFilter(Queue<Quaternion> quaternions, Quaternion lastMedian)
    	{
    		Quaternion median = new Quaternion(0, 0, 0, 0);
    		
    		foreach (Quaternion quaternion in quaternions)
    		{
    			float weight = 1 - (Quaternion.Dot(lastMedian, quaternion) / (Mathf.PI / 2)); // 0 degrees of difference => weight 1. 180 degrees of difference => weight 0.
    			Quaternion weightedQuaternion = Quaternion.Lerp(lastMedian, quaternion, weight);
    
    			median.x += weightedQuaternion.x;
    			median.y += weightedQuaternion.y;
    			median.z += weightedQuaternion.z;
    			median.w += weightedQuaternion.w;
    		}
    		
    		median.x /= quaternions.Count;
    		median.y /= quaternions.Count;
    		median.z /= quaternions.Count;
    		median.w /= quaternions.Count;
    		
    		return NormalizeQuaternion(median);
    	}
    	
    	public Quaternion NormalizeQuaternion(Quaternion quaternion)
    	{
    		float x = quaternion.x, y = quaternion.y, z = quaternion.z, w = quaternion.w;
    		float length = 1.0f / (w * w + x * x + y * y + z * z);
    		return new Quaternion(x * length, y * length, z * length, w * length);
    	}
    }

    Here you can find all the files I've used:

    https://www.dropbox.com/s/wbm0efgppt8236b/JointOrientationSmoothingSource.zip?dl=0

    (You will also need the Unity Plugin)

    Roberto Serrano - Diaple Networking

    Update: 

    Also with position smoothing.

    https://www.dropbox.com/s/4k81dvsi4vok9wa/JointSmoothingSource.zip?dl=0

    It's a simple algorithm as seen here: http://terpconnect.umd.edu/~toh/spectrum/Smoothing.html

    Monday, November 24, 2014 4:33 PM

All replies

  • Thanks for sharing
    Tuesday, November 25, 2014 5:47 AM
  • Hi

    thanks for this. 

    I was wondering if you could help or would know if the following script would work in Unity with modifications

    https://social.msdn.microsoft.com/Forums/en-US/850b61ce-a1f4-4e05-a0c9-b0c208276bec/joint-smoothing-code-for-c?forum=kinectv2sdk

    I'm new to it all and am a bit lost at the moment. I've tried attaching it to an empty object in the KinectView unity example MainScene but I don't see any difference?

    regards

    Paul

    Friday, November 28, 2014 12:47 AM
  • Hi Paul

    I'm planning on making a Joint position smooth filter next week (probably on monday). I will post the script here so you can give it a look.

    Regards

    Roberto Serrano - Diaple Networking

     

    Friday, November 28, 2014 10:05 AM
  • Thanks heaps Robert

    will look forward to implementing it next week

    have a great weekend

    Paul

    Saturday, November 29, 2014 12:56 AM
  • I have added the new link in the description. To try it just add the BodySmoothController script to an empty GameObject. It's not the same algorithm that the one posted in c++, this one is simpler, it's described here: http://terpconnect.umd.edu/~toh/spectrum/Smoothing.html

    Hope it helps and good luck.

    Roberto Serrano - Diaple Networking


    • Edited by r8berto Monday, December 01, 2014 9:30 AM
    Monday, December 01, 2014 9:24 AM
  • Thanks Roberto

    These are proving quite fun to play around with.

    Trying to get some meaningful data out of the ankle rotation and combinations of the above scripts are proving very useful

    cheers

    Paul

    Thursday, December 04, 2014 2:35 AM
  • I want to use it for hand rotation on my model, How should I use it? its giving weird rotations, if I apply directly
    Sunday, March 12, 2017 8:55 AM
  • Are you using the tip joints perhaps? Those usually have no rotation data. Might be messing up your results.
    Monday, March 13, 2017 8:46 AM