locked
Kinect Chinese/Japanese/Spanish Language Pack? RRS feed

  • Question

  • Hi,

    I am playing with the demo Speech app, and the the blue/green/red example is a good demo on the speech recognition example, though not great. What are some of the challenges that is facing the Kinect team to improve in speech recognition?

    Also, are there other language packs? It would be totally awesome to be able to try Chinese, Japanese, or Spanish with the Kinect.

     

    Thanks.
    Duncan

    Saturday, July 2, 2011 7:18 AM

Answers

  • Duncan,

    The core speech recognition technology used in Kinect SDK Beta samples is actually brought to you by the Microsoft Speech team, so it's hard for the Kinect team to give any details about challenges involved in speech recognition in general. You can find the development center for speech technologies here: http://msdn.microsoft.com/en-us/speech/default.aspx.

    The Kinect-specific language packs do have to be trained for a combination of language characteristics and Kinect audio characteristics, so that takes some resources to do for each pack. Right now we only have the language pack available for English, but whenever we have more we will share them with community.

    Thanks for your interest,
    Eddy


    I'm here to help
    Thursday, July 7, 2011 10:18 PM

All replies

  • Duncan,

    The core speech recognition technology used in Kinect SDK Beta samples is actually brought to you by the Microsoft Speech team, so it's hard for the Kinect team to give any details about challenges involved in speech recognition in general. You can find the development center for speech technologies here: http://msdn.microsoft.com/en-us/speech/default.aspx.

    The Kinect-specific language packs do have to be trained for a combination of language characteristics and Kinect audio characteristics, so that takes some resources to do for each pack. Right now we only have the language pack available for English, but whenever we have more we will share them with community.

    Thanks for your interest,
    Eddy


    I'm here to help
    Thursday, July 7, 2011 10:18 PM
  • How will we know when such language packs are released?  Is it possible to sign up for announcement of this sort?

    As an alternative, is it possible to use the array microphone features but feed them to the regular speech engine.  I realize the sound model would not be optimized for the kinect but it would allow us to use many of the essential features that are missing from the SDK speech. Such as

    - dictation

    - languages other than english

    - learning and multiple profiles

    I realize that I can use the Kinect as a regular microphone input devices, but it does not seem to provide any benefits of the array.  I supposed it is simple capturing as a stereo microphone?

    Wednesday, July 27, 2011 12:10 PM
  • jitterjames,

    There is currently no way to reliably sign up for these kinds of announcement, but I've given feedback to team about this and hopefully we can provide this for you soon. We do have a team twitter page (http://twitter.com/#!/KinectSDKTeam), but that probably doesn't have announcements at the level of granularity you would prefer.

    About using the "regular speech engine" I'm guessing you mean System.Speech namespace rather than Microsoft.Speech namespace, if you look through forum posts you will see that some community members have indeed gone with this solution and got it working, but this solution will result in a larger error rate without the optimized kinect language pack, and we are not providing support for solutions built using System.Speech namespace.

    You are right that if you use Kinect as a regular microphone input device, you will not get any benefits from the array. You will actually get 4 channels of raw audio from this microphone if you use WASAPI API for capturing, together with MMDevice API to enumerate capture devices. Look at AudioCaptureRaw sample (installed under C:\Users\Public\Documents\Microsoft Research KinectSDK Samples\Audio\AudioCaptureRaw\CPP).

    Thanks for your feedback,
    Eddy


    I'm here to help
    Wednesday, July 27, 2011 9:38 PM