none
FaceFrameSource Constructor in Unity RRS feed

  • Question

  • Hi! I have just installed the new Kinect v2.0 SDK and updated the Kinect for Unity package. I'm trying to use the FaceTracking features but the constructor for the FaceFrameSource Class is diferent in Unity.

    http://msdn.microsoft.com/en-us/library/microsoft.kinect.face.faceframesource.aspx

    It should require these three parameters but in Unity it only accepts one parameter, and it's a System.IntPtr.

    Thanks.

    -Roberto Serrano

    Friday, October 24, 2014 7:46 AM

Answers

  • Use the CreateSource function to create the source. This should get you started:

    using UnityEngine;
    using System.Collections;
    using System.Collections.Generic;
    using Windows.Kinect;
    using Microsoft.Kinect.Face;
    
    public class FaceSourceManager : MonoBehaviour
    {
    	private KinectSensor kinectSensor;
    	private int bodyCount;
    	private Body[] bodies;
    	private FaceFrameSource[] faceFrameSources;
    	private FaceFrameReader[] faceFrameReaders;
    
    	void Start()
    	{
    		// one sensor is currently supported
    		kinectSensor = KinectSensor.GetDefault();
    		
    		// set the maximum number of bodies that would be tracked by Kinect
    		bodyCount = kinectSensor.BodyFrameSource.BodyCount;
    		
    		// allocate storage to store body objects
    		bodies = new Body[bodyCount];
    		
    		// specify the required face frame results
    		FaceFrameFeatures faceFrameFeatures =
    			FaceFrameFeatures.BoundingBoxInColorSpace
    				| FaceFrameFeatures.PointsInColorSpace
    				| FaceFrameFeatures.BoundingBoxInInfraredSpace
    				| FaceFrameFeatures.PointsInInfraredSpace
    				| FaceFrameFeatures.RotationOrientation
    				| FaceFrameFeatures.FaceEngagement
    				| FaceFrameFeatures.Glasses
    				| FaceFrameFeatures.Happy
    				| FaceFrameFeatures.LeftEyeClosed
    				| FaceFrameFeatures.RightEyeClosed
    				| FaceFrameFeatures.LookingAway
    				| FaceFrameFeatures.MouthMoved
    				| FaceFrameFeatures.MouthOpen;
    		
    		// create a face frame source + reader to track each face in the FOV
    		faceFrameSources = new FaceFrameSource[bodyCount];
    		faceFrameReaders = new FaceFrameReader[bodyCount];
    		for (int i = 0; i < bodyCount; i++)
    		{
    			// create the face frame source with the required face frame features and an initial tracking Id of 0
    			faceFrameSources[i] = FaceFrameSource.Create(kinectSensor, 0, faceFrameFeatures);
    			
    			// open the corresponding reader
    			faceFrameReaders[i] = faceFrameSources[i].OpenReader();
    		}
    	}
    	
    	void Update()
    	{
    		// get bodies either from BodySourceManager object get them from a BodyReader
    		//var bodySourceManager = bodyManager.GetComponent<BodySourceManager>();
    		//bodies = bodySourceManager.GetData();
    		if (bodies == null)
    		{
    			return;
    		}
    		
    		// iterate through each body and update face source
    		for (int i = 0; i < bodyCount; i++)
    		{
    			// check if a valid face is tracked in this face source				
    			if (faceFrameSources[i].IsTrackingIdValid)
    			{
    				using(FaceFrame frame = faceFrameReaders[i].AcquireLatestFrame())
    				{
    					if(frame != null)
    					{
    						if (frame.TrackingId == 0)
    						{
    							continue;
    						}
    
    						// do something with result
    						var result = frame.FaceFrameResult;
    					}
    				}
    			}
    			else
    			{
    				// check if the corresponding body is tracked 
    				if (bodies[i].IsTracked)
    				{
    					// update the face frame source to track this body
    					faceFrameSources[i].TrackingId = bodies[i].TrackingId;
    				}
    			}
    		}		
    		
    	}
    }


    Carmine Sirignano - MSFT

    Friday, October 24, 2014 9:34 PM

All replies

  • Use the CreateSource function to create the source. This should get you started:

    using UnityEngine;
    using System.Collections;
    using System.Collections.Generic;
    using Windows.Kinect;
    using Microsoft.Kinect.Face;
    
    public class FaceSourceManager : MonoBehaviour
    {
    	private KinectSensor kinectSensor;
    	private int bodyCount;
    	private Body[] bodies;
    	private FaceFrameSource[] faceFrameSources;
    	private FaceFrameReader[] faceFrameReaders;
    
    	void Start()
    	{
    		// one sensor is currently supported
    		kinectSensor = KinectSensor.GetDefault();
    		
    		// set the maximum number of bodies that would be tracked by Kinect
    		bodyCount = kinectSensor.BodyFrameSource.BodyCount;
    		
    		// allocate storage to store body objects
    		bodies = new Body[bodyCount];
    		
    		// specify the required face frame results
    		FaceFrameFeatures faceFrameFeatures =
    			FaceFrameFeatures.BoundingBoxInColorSpace
    				| FaceFrameFeatures.PointsInColorSpace
    				| FaceFrameFeatures.BoundingBoxInInfraredSpace
    				| FaceFrameFeatures.PointsInInfraredSpace
    				| FaceFrameFeatures.RotationOrientation
    				| FaceFrameFeatures.FaceEngagement
    				| FaceFrameFeatures.Glasses
    				| FaceFrameFeatures.Happy
    				| FaceFrameFeatures.LeftEyeClosed
    				| FaceFrameFeatures.RightEyeClosed
    				| FaceFrameFeatures.LookingAway
    				| FaceFrameFeatures.MouthMoved
    				| FaceFrameFeatures.MouthOpen;
    		
    		// create a face frame source + reader to track each face in the FOV
    		faceFrameSources = new FaceFrameSource[bodyCount];
    		faceFrameReaders = new FaceFrameReader[bodyCount];
    		for (int i = 0; i < bodyCount; i++)
    		{
    			// create the face frame source with the required face frame features and an initial tracking Id of 0
    			faceFrameSources[i] = FaceFrameSource.Create(kinectSensor, 0, faceFrameFeatures);
    			
    			// open the corresponding reader
    			faceFrameReaders[i] = faceFrameSources[i].OpenReader();
    		}
    	}
    	
    	void Update()
    	{
    		// get bodies either from BodySourceManager object get them from a BodyReader
    		//var bodySourceManager = bodyManager.GetComponent<BodySourceManager>();
    		//bodies = bodySourceManager.GetData();
    		if (bodies == null)
    		{
    			return;
    		}
    		
    		// iterate through each body and update face source
    		for (int i = 0; i < bodyCount; i++)
    		{
    			// check if a valid face is tracked in this face source				
    			if (faceFrameSources[i].IsTrackingIdValid)
    			{
    				using(FaceFrame frame = faceFrameReaders[i].AcquireLatestFrame())
    				{
    					if(frame != null)
    					{
    						if (frame.TrackingId == 0)
    						{
    							continue;
    						}
    
    						// do something with result
    						var result = frame.FaceFrameResult;
    					}
    				}
    			}
    			else
    			{
    				// check if the corresponding body is tracked 
    				if (bodies[i].IsTracked)
    				{
    					// update the face frame source to track this body
    					faceFrameSources[i].TrackingId = bodies[i].TrackingId;
    				}
    			}
    		}		
    		
    	}
    }


    Carmine Sirignano - MSFT

    Friday, October 24, 2014 9:34 PM
  • It works like a charm. Thanks a lot!

    - Roberto Serrano.

     
    Monday, October 27, 2014 8:51 AM
  • this is for faceframe but how to get faceshapeanimations?
    Friday, October 31, 2014 5:55 PM
  • This is exactly what I need, but it seems that Microsoft.Kinect.Face is not available in unity. After some further research it looks like there's a difference in the .net Framework each uses. Can you provide any insight on this? is there another way to make this work?
    Sunday, November 9, 2014 11:49 PM
  • Be sure you have added both the Kinect.2.0 and the Kinect.Face unityPackages to your scene before calling the API's. The Kinect v2 packages are available here:

    http://go.microsoft.com/fwlink/?LinkID=513177


    Carmine Sirignano - MSFT

    Monday, November 10, 2014 10:28 PM
  • Can you give an example using HighDefinitionFaceFrame ?
    Tuesday, November 25, 2014 2:19 PM
  • Tuesday, November 25, 2014 9:28 PM
  • I am a beginner to unity3d and I want to know how to use this script?
    I tried to add script to a sphere but it prompts the null reference to object error?
    Can you please help me?

    waiting for your reply, thanks in advance :)

    • Proposed as answer by Saghir Rizvi Saturday, December 6, 2014 8:12 PM
    • Unproposed as answer by Saghir Rizvi Saturday, December 6, 2014 8:12 PM
    Saturday, December 6, 2014 11:25 AM
  • Dear Carmine,

             Thanks to read this reply.

             I met a probem using the FaceFrameSoure.create(senor,id, featrue) fucntion in unity-plugin.

              FaceFrameSoure.create(senor,id, featrue) can return faceFrameSoure and get face sucessfully in

             debug mode in Unity. But when I built the .exe file and run the .exe ,  FaceFrameSoure.create(senor,id,     featrue)  just return null.    

             I texted out some strings in the monior when got the null return  calling this function. 

              I don't know where goes wrong.  

              Please help me about this.   Thanks a lot, and thanks advance.  

    Sunday, November 15, 2015 4:28 AM