Driving existing face or mask mesh from Kinect

Question

I am working with Unity and the Kinect v2 SDK. My goal is to use the data from the Kinect sensor to control facial expressions on an EXISTING face or mask mesh.

I know that it is possible to create dynamic mesh data from the Kinect, but in my case I want to use an existing mesh.

Can anyone possibly point me to any examples or tutorials that might help?

Answer 1

I can't point you to specific examples and I haven't worked with HD face tracking module but I guess you could check what joints the face module can track and use their positions/rotations to drive a skinned mesh. Meaning you'd have to skin your existing mesh.

If you consider copying motions on the mesh vertex layer, I don't know if it can work. You'd have to somehow classify the mesh in areas in both meshes, associate them and try to copy transformations from one to the other.