none
Using Echo Cancellation with speech.cs example? RRS feed

  • Question

  • Hello,

    Does anyone have an example of using echo cancellation with the speech.cs example?  Or could someone at least get me started with adding echo cancellation to the speech example.

    I am working on starting tasks with voice commands, but some of those tasks play sounds over my speakers. I don't want those sounds to inadvertently turn on another task.

    I was trying to combine both examples some how but I am not very versed in C++ (which the echo cancellation is in).

    Thanks for any help!

     

    -Mike

    Wednesday, July 27, 2011 9:23 PM

Answers

  • Mike, I wrote a forum post about how to configure AEC in C#: http://social.msdn.microsoft.com/Forums/en-US/kinectsdkaudioapi/thread/a73396e9-2076-4de9-bdf6-17771ade3034

    does this address your problem? Let me know if you need anything else.
    Eddy


    I'm here to help
    Wednesday, July 27, 2011 9:42 PM
  • Just cast info.DeviceIndex to a short as Steve_118x suggested. KinectAudioSource.MicrophoneIndex is a short, and the value returned by info.DeviceIndex will be in the correct numeric range. Sorry for the confusion.

    About "the speech example still runs but doesn't pick up any sound", do you mean that the speech recognizer stops recognizing any speech you say? It's hard to say exactly why, because the speech recognizer just traps exceptions that might have been generated during sound processing, but I'm guessing that the SpeechRecognitionEngine is running into exceptions when reading from the Stream returned by KinectAudioSource.Start().

    For AEC to work, sound needs to be playing out of the speaker device specified as KinectAudioSource.SpeakerIndex, or else reading the stream will throw an exception--sorry, this is a limitation of the Kinect SDK Beta--so this is what I'm guessing is happening. Try playing a song through speakers before starting the AEC-configured speech recognition and see if that works better for you.

    Eddy


    I'm here to help
    • Marked as answer by Mike-ro-soft Wednesday, August 3, 2011 4:05 PM
    Friday, July 29, 2011 9:24 PM

All replies

  • Mike, I wrote a forum post about how to configure AEC in C#: http://social.msdn.microsoft.com/Forums/en-US/kinectsdkaudioapi/thread/a73396e9-2076-4de9-bdf6-17771ade3034

    does this address your problem? Let me know if you need anything else.
    Eddy


    I'm here to help
    Wednesday, July 27, 2011 9:42 PM
  • Wow that was much simpler than I thought.

    A couple of clarification questions about your example:

    short micIndex = -1; // (micIndex is a short? gives error)
                    foreach (AudioDeviceInfo info in source.FindCaptureDevices())
                    {
                        micIndex =  info.DeviceIndex;
                        break;
                    }
                    source.MicrophoneIndex = micIndex;
                    source.SpeakerIndex = 0;

                    Stream audioStream = source.Start();

    First, it throws an error when I try and build because micIndex is a short and info.DeviceIndex is a int. I casted info.DeviceIndex into a short, but that might wreck something I am not sure.

    Second, I have been playing around with adding your example to the speech example. But I have noticed as soon as I turn AEC on (SystemMode.OptibeamArrayAndAec) the speech example still runs but doesn't pick up any sound. Why is this?

    Any suggestions?

     

    -Mike



    Thursday, July 28, 2011 3:53 PM
  •  

     you can use a cast

    micIndex = (short)info.DeviceIndex;

    Steve.

    Thursday, July 28, 2011 4:56 PM
  • Just cast info.DeviceIndex to a short as Steve_118x suggested. KinectAudioSource.MicrophoneIndex is a short, and the value returned by info.DeviceIndex will be in the correct numeric range. Sorry for the confusion.

    About "the speech example still runs but doesn't pick up any sound", do you mean that the speech recognizer stops recognizing any speech you say? It's hard to say exactly why, because the speech recognizer just traps exceptions that might have been generated during sound processing, but I'm guessing that the SpeechRecognitionEngine is running into exceptions when reading from the Stream returned by KinectAudioSource.Start().

    For AEC to work, sound needs to be playing out of the speaker device specified as KinectAudioSource.SpeakerIndex, or else reading the stream will throw an exception--sorry, this is a limitation of the Kinect SDK Beta--so this is what I'm guessing is happening. Try playing a song through speakers before starting the AEC-configured speech recognition and see if that works better for you.

    Eddy


    I'm here to help
    • Marked as answer by Mike-ro-soft Wednesday, August 3, 2011 4:05 PM
    Friday, July 29, 2011 9:24 PM
  • Eddy -

    Okay, so you were correct. I thought I had a sound palying but this time I made sure I had something started first and it ran!

    However, the speech recognizer is still thinking that the music is speech. So I must be implementing it incorrectly.

    I am guessing form the above example that "audioStream" is the filtered output (stripped of music) from the microphone?  is that correct?

    I then take that filtered output and instead of starting a new stream in the speech example I feed in "audioStream" instead:

    " using (Stream s = audioStream)

    { sre.SetInputToAudioStream(s,  ..... " and so on.

    Does that seem correct?

     

    -Mike

    Tuesday, August 2, 2011 5:52 PM
  • yes, that seems correct. To try to check that the AEC config is doing anything at all for you pre-speech recognition, you could try to output recognized audio clips to file (as shown by DumpRecordedAudio method in Program.cs in Speech sample) both with and without AEC configuration turned on and see how much of the background music is audible in one (AEC on) vs the other (AEC off).

    Eddy


    I'm here to help
    Tuesday, August 2, 2011 8:30 PM