none
Simplest Head Pose code RRS feed

  • Question

  • Hi,

    I am interested in writing a program in order to get the data from IFTResult::Get3DPose and print it to the console, specifically x, y, and z as well as roll, pitch, and yaw.

    I was looking at the example files but was overwhelmed by their complexity being new to C++ and C# (I am willing to work with or learn either, however). I don't need any sophisticated video output, but simply the numerical values above.

    An advice on how to get started would be most appreciated. What is the minimum amount of overhead needed to obtain the head pose information? Do I need to initialize all sensors?

    Thanks!

    Tuesday, November 27, 2012 9:50 PM

All replies

  • I have spent the last few days familiarizing myself with Visual Studio and C# (I am primarily a Matlab programmer), so hopefully I can define my problem a little better:

    All of the sample code that comes with the SDK is made with WPF or something similar to display the camera output. I am interested in creating a simple console application and am interested in the simplest code to instantiate a face tracker and then query it for head pose data. I am hoping this can be done in a single main file or maybe one additional cs file.

    I am interested in getting the head pose data for each frame in order to track the movement of the head as fast as possible. The intention is to use the data for academic research on the use of head movements in gathering spatial data about the environment.

    Any help in this matter would be greatly appreciated!

    Thanks!

    Thursday, November 29, 2012 7:14 PM
  • The complexity of the samples is mainly to deal with visualization. It is best to start with a simple WinForm or WPF application that way you get the event system that make it easier to get the data from the sensor to pass to FaceTracking. Toolkit.FaceTracking provides the base objects you will want ot use. Have a look at FaceTrackingBasics-WPF sample as it demostrates getting the data from the camera and passing it to these objects.

    /// <summary>
    /// Updates the face tracking information for this skeleton
    /// </summary>
    internal void OnFrameReady(KinectSensor kinectSensor, ColorImageFormat colorImageFormat, byte[] colorImage, DepthImageFormat depthImageFormat, short[] depthImage, Skeleton skeletonOfInterest)
    
    ... 
    
    if (this.faceTracker != null)
    {
        FaceTrackFrame frame = this.faceTracker.Track(
            colorImageFormat, colorImage, depthImageFormat, depthImage, skeletonOfInterest);
    
        this.lastFaceTrackSucceeded = frame.TrackSuccessful;
        if (this.lastFaceTrackSucceeded)
        {
            if (faceTriangles == null)
            {
                // only need to get this once.  It doesn't change.
                faceTriangles = frame.GetTriangles();
            }
    
            this.facePoints = frame.GetProjected3DShape();
        }
    }


    Thursday, January 3, 2013 12:20 AM
  • I also face this problem. Have you solved it? 

    Give me some advise. Thank you.

    Wednesday, January 16, 2013 8:15 AM
  • The following is my understanding of the structure of head pose information.

    In general, in the demo FaceTrackingBasics-WPF, the FaceTrackingViewer use the SkeletonFaceTracker, and the SkeletonFaceTracker use the FaceTrackFrame. That is, the FaceTrackingViewer uses a skeletonFaceTracker to track the face. The SkeletonFaceTracker would get the 3D information from the FaceTrackFrame. 

    (FaceTrackingViewer --> SkeletonFaceTracker --> FaceTrackFrame) 

    In the FaceTrackFrame class, there is a vairable Rotation. I think this is what we needed for the head pose information.

            /// <summary>
            /// Rotation around X, Y, Z axes
            /// </summary>
            public Vector3DF Rotation
            {
                get
                {
                    this.CheckPtrAndThrow();
                    float scale;
                    Vector3DF rotationXyz;
                    Vector3DF translationXyz;
                    this.faceTrackingResultPtr.Get3DPose(out scale, out rotationXyz, out translationXyz);
                    return rotationXyz;
                }
            }


    1. Therefore I define a variable rotation in the SkeletonFaceTracker.

                private Vector3DF rotation;
                public Vector3DF getRotation()
                {
                    return rotation;
                }

    2. Everytime the face information is updated, I would update the rotation information.

                        if (this.lastFaceTrackSucceeded)
                        {
                            if (faceTriangles == null)
                            {
                                faceTriangles = frame.GetTriangles();
                            }

                            this.facePoints = frame.GetProjected3DShape();

                            // update the rotation
                            rotation = frame.Rotation;                       
                        }

    3. Everytime the colorImageFrame is ready in the FaceTrackingViewer, I would call the getRotation() function to update the rotation information.


    I think if the relationships between FaceTrackingViewer, SkeletonFaceTracker, and FaceTrackFrame are clear, it is easy to modify the code to obtain the head pose information. All the above are my understanding of the head pose information, feel free to point out any mistakes, thank you!
    Monday, January 28, 2013 9:06 AM
  • But how do the Rotation.X (y,Z) values represent the rotation of the head? Is the returned float value scaled in degrees?

    Tuesday, February 5, 2013 1:20 PM
  • But how do the Rotation.X (y,Z) values represent the rotation of the head? Is the returned float value scaled in degrees?

    I believe that those are rotations around X, Y, Z axes. For example if you do Rotation.X you'll get the Pitch angle, then Y = Yaw and Z = Roll angle.

    However, I think that this is not working well for extreme angles, for example if the users turns his head too much to the left I believe that the face tracking won't succeed, so in terms this condition: "if (this.lastFaceTrackSucceeded)" won't be met.

    I believe that this will work nicely only for Roll angle (Rotation.Z), and for other angles there would need to be created a more complex method and it would be nice if somebody would provide it here as an answer.

    And yes, I believe that they are in degrees.

    • Edited by Miroslav Smukov Wednesday, February 6, 2013 1:00 AM didn't answered completely
    Wednesday, February 6, 2013 12:58 AM
  • does anybody know how to display the result on the kinect video?

    thanks

    Thursday, August 1, 2013 1:42 PM
  • do you did it?
    Friday, January 19, 2018 4:41 PM