none
Question about using face tracking sdk for asynchronous tracking. RRS feed

  • Question

  • I have a question that can the face tracking engine use the normal video as the input data? That means asynchronous tracking, first use the camera records the face images as  a video files (like .avi), and then use this video file as the input data to track the face feature points, can face tracking sdk do it?
    Thank you very much.
    • Edited by yusiyang Friday, October 19, 2012 1:49 AM
    Friday, October 19, 2012 1:49 AM

All replies

  • The current implementation of face tracking in the Kinect SDK uses a combination of the video frame, the depth frame, and (optionally) the skeleton frame.  I don't know of a way to perform face tracking with the SDK without using at least the video frame and depth frame together.
    • Proposed as answer by kengr Tuesday, October 30, 2012 1:59 AM
    Tuesday, October 30, 2012 1:59 AM
  • The current implementation of face tracking in the Kinect SDK uses a combination of the video frame, the depth frame, and (optionally) the skeleton frame.  I don't know of a way to perform face tracking with the SDK without using at least the video frame and depth frame together.
    Thank you very much for your information. Actually I do use the video frame and the depth frame, what I want to say is first record the video frame and the depth frame save them in the hard disk, and then use the saved frame image(both the video and depth) as the input data to pass them to the face tracking engine to do face tracking. I want to know whether this method can work or not.
    Wednesday, November 7, 2012 3:17 AM
  • Based on what I've seen in C#, the face tracking algorithm takes the frame data as input (i.e., it doesn't read directly from the Kinect device, but relies on the application code to get all the device data and then pass it into the face tracking API).  So, although I haven't tried it, I think it should be possible to write the frame data (video, depth, and possibly skeleton) to a file, and then later read the data from the file and send it to the face tracking algorithm.  Of course, the saving of data would have to keep up with the incoming frame rate, to avoid dropping frames.  Whatever file format you define, when reading it back you would need to reconstruct the frame data into its original forms (separate arrays) before passing the data into the face tracking API.  I don't know if the face tracking algorithm is affected by frame rate, but if it is, you would also need to control the timing to feed the saved frame data to face tracking at the appropriate rate.

    Another approach you might consider is using the Kinect Studio application to record Kinect frame data to a file, and then use Kinect Studio to later play that data back into your application.  Kinect Studio gets between the device and the application, so it can play back the saved data to the app as if the data were coming from the Kinect device.  In this scenario, your application would be written to receive frames from the Kinect streams, but it would actually be receiving saved frame data from Kinect Studio.


    • Proposed as answer by kengr Wednesday, November 7, 2012 4:48 AM
    • Edited by kengr Wednesday, November 7, 2012 3:16 PM correct typo
    Wednesday, November 7, 2012 4:47 AM
  • Based on what I've seen in C#, the face tracking algorithm takes the frame data as input (i.e., it doesn't read directly from the Kinect device, but relies on the application code to get all the device data and then pass it into the face tracking API).  So, although I haven't tried it, I think it should be possible to write the frame data (video, depth, and possibly skeleton) to a file, and then later read the data from the file and send it to the face tracking algorithm.  Of course, the saving of data would have to keep up with the incoming frame rate, to avoid dropping frames.  Whatever file format you define, when reading it back you would need to reconstruct the frame data into its original forms (separate arrays) before passing the data into the face tracking API.  I don't know if the face tracking algorithm is affected by frame rate, but if it is, you would also need to control the timing to feed the saved frame data to face tracking at the appropriate rate.

    Another approach you might consider is using the Kinect Studio application to record Kinect frame data to a file, and then use Kinect Studio to later play that data back into your application.  Kinect Studio gets between the device and the application, so it can play back the saved data to the app as if the data was coming from the Kinect device.  In this scenario, your application would be written to receive frames from the Kinect streams, but it would actually be receiving saved frame data from Kinect Studio.

    A thousand thanks to you! I will try.
    Wednesday, November 7, 2012 5:53 AM