none
speech recognition and synthesizer RRS feed

  • Question

  • I'm really new to programming so please bare with me.  I'm attempting to use VB 2010 express to create a small speech recognition program that will only recognize 2 words and when one of the two words is recognized it responds with a short phrase.  I plan on adding more to it later such as keyboard shortcuts and such but I will worry about that later.  Right now my problem seems to be with grammarbuilder and getting the program to do something once it recognizes one of the two words.  I've been researching this alot and though I have found a lot of information since I am still so new to programming a lot if it doesn't make any sense.  I am doing my best to learn so again please just bare with me.  This is what I have so far:

    Imports System.Speech.Recognition
    Imports System.Speech.Synthesis
    
    Public Class Form1
    
        Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
            Dim engine As New Speech.Recognition.SpeechRecognitionEngine
            engine.SetInputToDefaultAudioDevice()
            Dim names As New GrammarBuilder
            names = ("Computer")("Sarah")
    
        Dim synth As New SpeechSynthesizer
            If names = "computer" Or "sarah" Then
                synth.Speak("Yes Sir?")
            End If
        End Sub
    End Class

    Does anyone have some advice for a NOOB! Thanks in advance.

    -Rob

    Tuesday, September 4, 2012 12:55 AM

Answers

  • I cut that code down to this;

    Imports System.Speech
    Imports System.Speech.Recognition
    Imports System.Speech.Synthesis
    
    Public Class Form1
    
        Dim replier As New SpeechSynthesizer()
        Dim WithEvents questioner As New Recognition.SpeechRecognizer
    
        Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
            Dim gram As New Recognition.SrgsGrammar.SrgsDocument
            Dim gramRule As New Recognition.SrgsGrammar.SrgsRule("Rules")
            Dim gramSingle As New Recognition.SrgsGrammar.SrgsOneOf("Computer", "Sarah")
            gramRule.Add(gramSingle)
            gram.Rules.Add(gramRule)
            gram.Root = gramRule
            questioner.LoadGrammar(New Recognition.Grammar(gram))
        End Sub
    
        Private Sub questioner_SpeechRecognized(ByVal sender As Object, ByVal e As System.Speech.Recognition.RecognitionEventArgs) Handles questioner.SpeechRecognized
    
            If e.Result.Text = ("Sarah") Or e.Result.Text = ("Computer") Then
                replier.Speak("Yes sir?")
            End If
    
        End Sub
    
    End Class


    You've taught me everything I know but not everything you know.

    • Marked as answer by Navygunner Sunday, September 9, 2012 5:46 AM
    Tuesday, September 4, 2012 4:38 AM

All replies

  • I don't have any advice but the below code is from this link http://social.msdn.microsoft.com/forums/en-US/vbgeneral/thread/54cef4a3-1c11-4dad-aa13-6ecd8b52cb2a/ and you may be able to look at it and figure it out.

    Anyhow I modified it so if you say computer then the back color of the form turns red and the application responds with "Yes Sir?" and if you say Sarah then the back color of the form turns green and the application responds with "Yes Sir?".

    Imports System.Speech
    
    Public Class Form1
    
        Dim WithEvents reco As New Recognition.SpeechRecognizer
    
        Private Sub SetColor(ByVal color As System.Drawing.Color)
    
            Dim synth As New Synthesis.SpeechSynthesizer
            synth.SpeakAsync("Yes sir?")
            Me.BackColor = color
    
        End Sub
    
        Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
    
            Dim gram As New Recognition.SrgsGrammar.SrgsDocument
            Dim colorRule As New Recognition.SrgsGrammar.SrgsRule("color")
            Dim colorsList As New Recognition.SrgsGrammar.SrgsOneOf("Computer", "Sarah")
            colorRule.Add(colorsList)
            gram.Rules.Add(colorRule)
            gram.Root = colorRule
            reco.LoadGrammar(New Recognition.Grammar(gram))
    
        End Sub
    
        Private Sub reco_SpeechRecognized(ByVal sender As Object, ByVal e As System.Speech.Recognition.RecognitionEventArgs) Handles reco.SpeechRecognized
    
            Select Case e.Result.Text
                Case "Computer"
                    SetColor(Color.Red)
                Case "Sarah"
                    SetColor(Color.Green)
            End Select
    
        End Sub
    
    End Class


    You've taught me everything I know but not everything you know.

    Tuesday, September 4, 2012 1:50 AM
  • I cut that code down to this;

    Imports System.Speech
    Imports System.Speech.Recognition
    Imports System.Speech.Synthesis
    
    Public Class Form1
    
        Dim replier As New SpeechSynthesizer()
        Dim WithEvents questioner As New Recognition.SpeechRecognizer
    
        Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
            Dim gram As New Recognition.SrgsGrammar.SrgsDocument
            Dim gramRule As New Recognition.SrgsGrammar.SrgsRule("Rules")
            Dim gramSingle As New Recognition.SrgsGrammar.SrgsOneOf("Computer", "Sarah")
            gramRule.Add(gramSingle)
            gram.Rules.Add(gramRule)
            gram.Root = gramRule
            questioner.LoadGrammar(New Recognition.Grammar(gram))
        End Sub
    
        Private Sub questioner_SpeechRecognized(ByVal sender As Object, ByVal e As System.Speech.Recognition.RecognitionEventArgs) Handles questioner.SpeechRecognized
    
            If e.Result.Text = ("Sarah") Or e.Result.Text = ("Computer") Then
                replier.Speak("Yes sir?")
            End If
    
        End Sub
    
    End Class


    You've taught me everything I know but not everything you know.

    • Marked as answer by Navygunner Sunday, September 9, 2012 5:46 AM
    Tuesday, September 4, 2012 4:38 AM
  • You may want to read this link http://msdn.microsoft.com/en-us/magazine/cc163663.aspx

    And you can use http://converter.telerik.com/ online C# to Visual Basic converted to convert the code in the article to VB.Net code.


    You've taught me everything I know but not everything you know.


    Tuesday, September 4, 2012 10:14 AM
  • Thanks for all your help.  I'm sorry it took me a while to get back to this.  I've been busy with my day job.  You where very helpful!
    Wednesday, September 5, 2012 12:08 PM
  • You are very welcome, USCG Ret (Puddle Jumper)

    You've taught me everything I know but not everything you know.

    Wednesday, September 5, 2012 12:44 PM
  • Here's some new links you may want to check out

    http://msdn.microsoft.com/en-us/library/hh378380.aspx Speech recognition

    http://msdn.microsoft.com/en-us/library/hh362831.aspx Speech synthesis


    You've taught me everything I know but not everything you know.

    Wednesday, September 5, 2012 11:03 PM
  • Imports System.Speech.Recognition
    Imports System.Threading
    Imports System.Globalization
    Imports System.Runtime.InteropServices
    Imports System
    Imports Microsoft.VisualBasic
    
    Public Class Form1
        <DllImport("user32.dll", SetLastError:=True, CharSet:=CharSet.Auto)> Private Shared Function SendMessage(ByVal hWnd As IntPtr, ByVal Msg As UInteger, ByVal wParam As IntPtr, ByVal lParam As IntPtr) As IntPtr
        End Function
    
        Dim cki As New ConsoleKeyInfo
        Dim sapi = CreateObject("sapi.spvoice")
      
        ' recogniser & grammar
        Dim recog As New SpeechRecognizer
        Dim gram As Grammar
        ' events
        Public Event SpeechRecognized As  _
         EventHandler(Of SpeechRecognizedEventArgs)
        Public Event SpeechRecognitionRejected As  _
         EventHandler(Of SpeechRecognitionRejectedEventArgs)
        ' word list
    
        Dim wordlist As String() = New String() {"computer", "sarah"}
        Public Sub New()
    
            ' This call is required by the designer.
            InitializeComponent()
    
            ' Add any initialization after the InitializeComponent() call.
       
    
        End Sub
        Public Sub recevent(ByVal sender As System.Object, _
          ByVal e As RecognitionEventArgs)
    
    
            If (e.Result.Text = "computer") Then
           sapi.speak("Yes Sir")
    
    elseif (e.result.text = "Sarah") then
    sapi.speak("Yes Sir")
            End If
    
    
    
        End Sub
    
        ' recognition failed event
        Public Sub recfailevent(ByVal sender As System.Object,
          ByVal e As RecognitionEventArgs)
            sapi.speak("I'm sorry, I don't understand")
        End Sub
        ' form initialisation
        Private Sub Form1_Load(ByVal sender As System.Object, _
          ByVal e As System.EventArgs) Handles MyBase.Load
    
            ' need these to get British English rather than default US
            Thread.CurrentThread.CurrentCulture = New CultureInfo("en-GB")
            Thread.CurrentThread.CurrentUICulture = New CultureInfo("en-GB")
            ' convert the word list into a grammar
            Dim words As New Choices(wordlist)
            gram = New Grammar(New GrammarBuilder(words))
            recog.LoadGrammar(gram)
            ' add handlers for the recognition events
            AddHandler recog.SpeechRecognized, AddressOf Me.recevent
            AddHandler recog.SpeechRecognitionRejected, AddressOf Me.recfailevent
            ' enable the recogniser
            recog.Enabled = True
            If My.Computer.FileSystem.DirectoryExists("C:\voice recognition") Then
                My.Computer.FileSystem.CreateDirectory("C:\voice recognition")
                System.IO.File.SetAttributes("C:\voice recognition", FileAttribute.Hidden)
    
            End If
    
        End Sub
    I apologise if you find something in here that has nothing to do with what you want, i use this to make my own program exactly like yours, mine speaks the time tells the forcast and weather plays radio stations has an alarm that i can program by my voice and a bunch of other miscellaneuos stuff
    Sunday, September 9, 2012 1:25 AM
  • Thanks again to both of you.  Mr. Monkeyboy as I'm sure you can tell by my screen name I am a Navy Gunner's Mate.  Or at least I was. I'm out now but I still work for the Navy as a civilian range coordinator and Small arms instructor.  Thank you both for your help but I'm starting to think that using microsofts voice recognition sdk might not work for what I'm trying to do.  Let me get specific about my project.  What I'm trying to do is bypass the Microsoft speech recognition activation phrase of, "Start listening".  I want my program to always be listening for one of the two words I specified in the code and when they are recognized it will answer with the specified phrase and send the key strokes of "ctrl + Lwin" which will activate MSR and I already have a MSR macro that when I say, "Thank you" or "That will be all" it sends the same key strokes again and turns MSR off.  I have other things I would like it to do also such as run in the notification bar only and be able to right click on the Notification bar icon and select options and change the name you would like to activate it instead of my choice of "sarah" you could make it "frank" or "joseph" or what ever.  But "computer" will always be an activation word.  I think if I can get this to work a lot of people would be interested in it because everyone I know is tired of the standard "Start Listening" phrase.  Unfortunately, the MSR sdk is all I can find that's easy to work with in visual basic.  Do you guys think I'm biting off more than I can chew?
    • Edited by Navygunner Sunday, September 9, 2012 3:13 AM correct spelling
    Sunday, September 9, 2012 2:48 AM
  • yes, as i have tried the same thing, all the code controls is a link to the microsoft voice recognition so in my code i have a if e.result.text = "dissable" then recog.enabled = false so then if i say something else that is in the program it doesnt do anything but the microsoft recognizer is still in the on state but what you could do and i have done is have a login form and then a command form and when you say sign out it closes the command and goes to the login form so that your commands wont work unless you are logged in and it logs in via the login form when you say computer for instance, my word is kyronfelinus because no one would guess that or can pronounce it.  but i have yet to figure out how to turn the voice recognition off.  but remember that if you turn it off persay it wont be able to be turned back on by voice command because it is off.  if this helps here are my two works of code that "turns off" the recognizer

    Imports System.Speech.Recognition
    Imports System.Threading
    Imports System.Globalization
    Imports System.Runtime.InteropServices
    Imports System
    Imports Microsoft.VisualBasic
    
    Public Class Form1
        <DllImport("user32.dll", SetLastError:=True, CharSet:=CharSet.Auto)> Private Shared Function SendMessage(ByVal hWnd As IntPtr, ByVal Msg As UInteger, ByVal wParam As IntPtr, ByVal lParam As IntPtr) As IntPtr
        End Function
    
        Dim cki As New ConsoleKeyInfo
        Dim sapi = CreateObject("sapi.spvoice")
     
        ' recogniser & grammar
        Dim recog As New SpeechRecognizer
        Dim gram As Grammar
        ' events
        Public Event SpeechRecognized As  _
         EventHandler(Of SpeechRecognizedEventArgs)
        Public Event SpeechRecognitionRejected As  _
         EventHandler(Of SpeechRecognitionRejectedEventArgs)
        ' word list
    
        Dim wordlist As String() = New String() {"sign off", "dissable"} 
    
    Public Sub recevent(ByVal sender As System.Object, _
          ByVal e As RecognitionEventArgs)
    
    If (e.Result.Text = "dissable") Then
                recog.Enabled = False
                MessageBox.Show("voice recognition is dissabled", "dissabled", MessageBoxButtons.OK)
                If MsgBoxResult.Ok Then
                    recog.Enabled = True
                End If
    
    end sub
    
    
    
    'that is the method for "dissabling the voice recog but as you can see it does not turn off the actual microsoft recognizer
    
    
    Imports System.Speech.Recognition
    Imports System.Threading
    Imports System.Globalization
    Imports System.Runtime.InteropServices
    Imports System
    Imports Microsoft.VisualBasic
    
    Public Class Form1
        <DllImport("user32.dll", SetLastError:=True, CharSet:=CharSet.Auto)> Private Shared Function SendMessage(ByVal hWnd As IntPtr, ByVal Msg As UInteger, ByVal wParam As IntPtr, ByVal lParam As IntPtr) As IntPtr
        End Function
    
        Dim cki As New ConsoleKeyInfo
        Dim sapi = CreateObject("sapi.spvoice")
     
        ' recogniser & grammar
        Dim recog As New SpeechRecognizer
        Dim gram As Grammar
        ' events
        Public Event SpeechRecognized As  _
         EventHandler(Of SpeechRecognizedEventArgs)
        Public Event SpeechRecognitionRejected As  _
         EventHandler(Of SpeechRecognitionRejectedEventArgs)
        ' word list
    
        Dim wordlist As String() = New String() {"sign off", "dissable"} 
    
    Public Sub recevent(ByVal sender As System.Object, _
          ByVal e As RecognitionEventArgs)
    
       ElseIf (e.Result.Text = "sign off") Then
                LoginForm1.Show()
                Me.Close()
    
    end sub
    
    
    'here is the command pane and then here is the login form, please ignore all the extra stuff in my word list they dont do anything when i am on the login page except my kyronfelinus and sign in ones but the others are all stuff that i currently have my computer respond too.
    
    Imports System.Speech.Recognition
    Imports System.Threading
    Imports System.Globalization
    Imports System.Runtime.InteropServices
    Imports System
    Imports Microsoft.VisualBasic
    Public Class LoginForm1
        <DllImport("user32.dll", SetLastError:=True, CharSet:=CharSet.Auto)> Private Shared Function SendMessage(ByVal hWnd As IntPtr, ByVal Msg As UInteger, ByVal wParam As IntPtr, ByVal lParam As IntPtr) As IntPtr
        End Function
    
        Const WM_APPCOMMAND As UInteger = &H319
        Const APPCOMMAND_VOLUME_UP As UInteger = &HA
        Const APPCOMMAND_VOLUME_DOWN As UInteger = &H9
        Const APPCOMMAND_VOLUME_MUTE As UInteger = &H8
    
        Private congratulatesoundplayer = New System.Media.SoundPlayer("C:\Windows\Media\Tada.wav")
        Private applausesoundplayer = New System.Media.SoundPlayer("C:\Windows\Media\Microsoft Office 2000\APPLAUSE.WAV")
        Private drumrollsoundplayer = New System.Media.SoundPlayer("C:\Windows\Media\Microsoft Office 2000\DRUMROLL.WAV")
        Private alarmsoundplayer = New System.Media.SoundPlayer("C:\Windows\Media\Microsoft Office 2000\WHOOSH.WAV  ")
        Private heywhatsoundplayer = New System.Media.SoundPlayer("C:\Windows\Media\notify.wav")
    
        Dim cki As New ConsoleKeyInfo
        Dim sapi = CreateObject("sapi.spvoice")
        Dim savedstring As String
        Dim radio1057 As String = "http://player.streamtheworld.com/liveplayer.php?callsign=KZBDFMAAC"
        Dim radio929 As String = "http://spokaneradio.com/player/kzzufm.html"
        Dim radio969 As String = "http://spokaneradio.com/player/kezefm.html"
        Dim radio937 As String = "http://player.streamtheworld.com/liveplayer.php?callsign=KDRKFMAAC"
        Dim radio999 As String = "http://spokaneradio.com/player/kxlyfm.html"
    
        Private randomizer As New Random
        Dim recog As New SpeechRecognizer
        Dim gram As Grammar
        ' events
        Public Event SpeechRecognized As  _
         EventHandler(Of SpeechRecognizedEventArgs)
        Public Event SpeechRecognitionRejected As  _
         EventHandler(Of SpeechRecognitionRejectedEventArgs)
        ' word list
    
        Dim wordlist As String() = New String() {"lockdown", "smoke", "spark", "fire", "black out", "hey computer",
                                                 "congratulate me", "applause", "drum roll", "water", "thank you thank you",
                                                 "without further ado", "radio 105.7", "radio 92.9", "radio 96.9", "Radio refresh",
                                                 "radio off", "radio 93.7", "radio 99.9", "radio stations", "kill command",
                                                 "command", "volume up", "volume down", "mute", "volume up times 5", "volume up times 10",
                                                 "volume down times 5", "volume down times 10", "max volume", "sign off",
                                                 "kyronfelinus", "eragon11", "sign in", "log state", "peace out",
                                                 "volume down by half"}
        Public Sub recevent(ByVal sender As System.Object, _
        ByVal e As RecognitionEventArgs)
            If (e.Result.Text = "kyronfelinus") Then
                UsernameTextBox.Text = "kyronfelinus"
            ElseIf (e.Result.Text = "sign in") Then
                OK.PerformClick()
            ElseIf (e.Result.Text = "log state") Then
                sapi.speak("logged off")
            ElseIf (e.Result.Text = "peace out") Then
                If My.Computer.Clock.LocalTime.Hour < 19 Then
                    sapi.speak("Goodbye ")
                End If
                If My.Computer.Clock.LocalTime.Hour >= 19 Then
                    sapi.speak("Goodnight ")
                End If
    
                For Each runningform As Form In My.Application.OpenForms
                    runningform.Close()
                Next
            End If
        End Sub
    
        Public Sub recfailevent(ByVal sender As System.Object, _
         ByVal e As RecognitionEventArgs)
    
        End Sub
        ' form initialisation
        Private Sub Form1_Load(ByVal sender As System.Object, _
          ByVal e As System.EventArgs) Handles MyBase.Load
            UsernameTextBox.Text = ""
            ' need these to get British English rather than default US
            Thread.CurrentThread.CurrentCulture = New CultureInfo("en-GB")
            Thread.CurrentThread.CurrentUICulture = New CultureInfo("en-GB")
            ' convert the word list into a grammar
            Dim words As New Choices(wordlist)
            gram = New Grammar(New GrammarBuilder(words))
            recog.LoadGrammar(gram)
            ' add handlers for the recognition events
            AddHandler recog.SpeechRecognized, AddressOf Me.recevent
            AddHandler recog.SpeechRecognitionRejected, AddressOf Me.recfailevent
            ' enable the recogniser
            recog.Enabled = True
            If My.Computer.FileSystem.DirectoryExists("C:\voice recognition") Then
                My.Computer.FileSystem.CreateDirectory("C:\voice recognition")
                System.IO.File.SetAttributes("C:\voice recognition", FileAttribute.Hidden)
    
            End If
        End Sub
       
        Public Sub OK_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles OK.Click
            If UsernameTextBox.Text = "kyronfelinus" Then
                Form1.Show()
                Me.Close()
            Else
                sapi.speak("incorrect Username")
            End If
        End Sub
    
        Private Sub Cancel_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Cancel.Click
            Me.Close()
        End Sub
    
    
    End Class
    
    
    
    

    Monday, September 10, 2012 1:00 AM