none
Kinect voice commands to control connected serial devives RRS feed

  • Question

  • Hello,

    I fairly new to working with Kinect.  I'm asking for anyone tp point me in the right direction for a forum to use Kinect, connect to a PC, to accept custom voice commands such as "TV off" TV on" TV Volume up '3'".  These devices, TV, receiver, etc... would be connected to the PC via serial connections with well documented serial commands from the manufacturer.

    What forum would this be?  Has anyone worked in something like this?

    Thank you for any help!

    PJCthoth

    Saturday, January 19, 2013 8:59 PM

All replies

  • Hello

    Yes, in kinect you have the Speech Recognition Component. I think you should go throug the Sample programs which microsoft had provided in their V1.6 sdk. Once you install the V1.6 sdk, the Kinect Studio will have list of programs which you have to install and run those to see your Kinect working. So, there are some examples on the Speech Recognition too. I suggest you better go through those. I also suggest, as you are new to Kinect, would recommend to go through some sample eBook which gives a good explanation on Kinect for all components.

    I Personally felt that the following Book gives you a good insight. Please do try this link.

    http://it-ebooks.info/book/656/

    Note: The eBooks are proprietory of the site and please do follow the instructions given in the site before you download.

    Thanks

    -Srini


    Srinivasa Nadella

    Monday, January 21, 2013 1:24 PM
  • Start from "Speech Basics" example from Kinect for Windows Developer Toolkit.

    If you want utilize American language, modify the SpeechGrammar.xml with your commands... for example:

    <grammar version="1.0" xml:lang="it-IT" root="rootRule" tag-format="semantics/1.0-literals" xmlns="http://www.w3.org/2001/06/grammar">
      <rule id="rootRule">
        <one-of>
          <item>
            <tag>OFF</tag>
            <one-of>
              <item> tv off </item>
              <item> tv shutdown </item>
              <item> tv power off </item>
            </one-of>
          </item>
          <item>
            <tag>PLUS_3</tag>
            <one-of>
              <item> tv volume up three </item>
            </one-of>
          </item>
         </one-of>
      </rule>
    </grammar>


    In the file MainWindow.xaml.cs modify the private void SpeechRecognized method with:

    private void SpeechRecognized(object sender, SpeechRecognizedEventArgs e)
    {
        // Speech utterance confidence below which we treat speech as if it hadn't been heard
        const double ConfidenceThreshold = 0.3;
     
        ClearRecognitionHighlights();
     
        if (e.Result.Confidence >= ConfidenceThreshold)
        {
            switch (e.Result.Semantics.Value.ToString())
            {
                case "OFF":
                    //ADD COMMANDS TO EXECUTE WHEN THE SYSTEM RECOGNIZE "tv off" or "tv shutdown" or "tv power off"
                    break;
     
                case "PLUS_3":
                    //ADD COMMANDS TO EXECUTE WHEN THE SYSTEM RECOGNIZE "tv volume up three"
                    break;
            }
        }
    }

    Now you can start to focus on the serial connection without lose time on the speech recognition ;)
    After you can easly modify the user interface... remember to delete the Turtle! :D

    • Proposed as answer by The Thinker Tuesday, January 29, 2013 3:42 PM
    Tuesday, January 29, 2013 1:38 AM
    Moderator
  • Do as clemente suggests except you can find many examples for accessing the serial ports on the vb  and c# forums and plug them into this code from his post above:

        switch (e.Result.Semantics.Value.ToString())
            {
                case "OFF":
                    //ADD COMMANDS TO EXECUTE WHEN THE SYSTEM RECOGNIZE "tv off" or "tv shutdown" or "tv power off"
                    break;
     
                case "PLUS_3":
                    //ADD COMMANDS TO EXECUTE WHEN THE SYSTEM RECOGNIZE "tv volume up three"
                    break;
            }

    in vb.net forums you will find plenty of serial port examples (their might be some in c# forums but the vb.net ones are simple to understand):

    http://social.msdn.microsoft.com/Forums/en-US/vbgeneral/threads?page=1

    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - "Sherlock holmes" "speak softly and carry a big stick" - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda. Blog - http://www.computerprofessions.co.nr


    • Edited by The Thinker Tuesday, January 29, 2013 3:41 PM
    Tuesday, January 29, 2013 3:41 PM