Detecting multiple people using Kinect audio source RRS feed

  • Question

  • Hello,

    I'm working with the Kinect for windows v2 and I want to detect multiple people using the Kinect audio frame source.

    I mean that if multiple people are talking simultaneously while the Kinect is working, then I want the Kinect to be able to distinguish between the people according to their voices and to save each voice to a separate audio file. What do I need to do to the Kinect audio frame source to achieve that?



    Friday, February 27, 2015 9:23 AM

All replies

  • The AudioBeamSubFrame will provide you access to the AudioBodyCorrelation object. You can review the AudioBasics sample to see how to get access to the AudioBeamSubFrame list.

    The API returns a collection of audio body correlations, in relation to the beam itself. You can use this to identify which enrolled tracked body is likely speaking. If Kinect is not tracking any bodies, this list will be empty. If one or more bodies is being tracked, this will return an ordered list containing body tracking IDs. The list is ordered based on the horizontal angle of the body relative to the current audio beam angle. The first body in the list is the one closest to the audio beam and the last is the furthest away from the audio beam.

    You will use the data against a list of tracked bodies you currently have. Refer to the body basics on how to get the tracking id's..

    Carmine Sirignano - MSFT

    Friday, February 27, 2015 6:47 PM
  • ... multiple people are talking simultaneously ...

    I understand Carmine response is about telling "which body is talking when only one talked".

    I don't think you can separate "voice sources" in an audio frame with several speakers. If you had the audio data from each internal micro in the array maybe you could use some signal processing method like blind separation.

    Saturday, February 28, 2015 9:40 AM
  • correct, if there are 2 tracked people in the same area, the api's will return a collection of body id's.

    Carmine Sirignano - MSFT

    Monday, March 2, 2015 7:41 PM