none
How to apply FaceShapeAnimations values without source FaceAlignment object RRS feed

  • Question

  • Hello!

    I am trying to transfer Kinect captured data to a remote PC over network. And I want to capture FaceAlignment.AnimationUnits and reconstruct face mesh by this points. It is very convenient way to not transfer whole face mesh points and animate characters face - just 16 float values VS 1.3k dots.

    But I cant find any analog for FaceModel.CalculateVerticesForAlignment(FaceAlignment faceAlignment) that applying AnimationUnits  to the face. And I cant find any way to prepare my own FaceAlignment object by recived points. 

    Is it exist any way to apply this points to face mesh? Or some source code for CalculateVerticesForAlignment or any relation data between face vertices and FaceShapeAnimations? Or any instructions how to transfer that data to another avatar with prepared face mimic bones? 

    Regards,

    Alex.

    Wednesday, February 8, 2017 12:38 PM

Answers

  • I have found solution with a little reverse engineering. As I realized FaceAlignment object contains data in plain way. I just inspected memory after nativePrt and found out that its look like:

    [some data (len = x6 floats)] [FaceOrientation (x4 floats)] [HeadPivotPoint (x3 floats)] [!REQUIRED AnimationUnits (17 floats)! ][some other data...

      So I can just rewrite this memory to setup my own FaceAlignment or copy. If anybody needs this code I just extended Unity wrapper for this class in C#: 

    public RootSystem.Collections.Generic.Dictionary<Microsoft.Kinect.Face.FaceShapeAnimations, float> AnimationUnits
    {
        get
        {
            //
            // Default Get code...
            //
        }
        set
        {
            if (value == null || _pNative == RootSystem.IntPtr.Zero)
                return;
    
            float[] data = new float[(int)FaceShapeAnimations.Count];
            for (int i = 0; i < (int)FaceShapeAnimations.Count; i++)
            {
                FaceShapeAnimations unit = (FaceShapeAnimations)i;
                data[i] = value.ContainsKey(unit) ? value[unit] : 0f;
            }
            SetAnimationUnits(data);
        }
    }
    public void SetAnimationUnits(float[] data)
    {
        if (data == null || data.Length != (int)FaceShapeAnimations.Count)
            return;
    
        uint offset = 6 + 4 + 3; // 6 - some other data, 4 - rotation, 3 - pivot
        var auPrt = (System.IntPtr)((ulong)_pNative + offset * sizeof(float));   // Offset
    
        System.Runtime.InteropServices.Marshal.Copy(data, 0, auPrt, data.Length);
    }

    • Marked as answer by Alex Zu Thursday, February 9, 2017 3:10 PM
    Thursday, February 9, 2017 3:09 PM