none
Help with ideas for sign language recogition to speech with kinect sdk

    Question

  • Im trying to reasearch how to turn an american sign language symbol from the hand into speech. For instance, the person signs the letter A and the program uses a text-to-speech engine to speak the word. Problem is figuring out how to do the below:

    1. Code to record the sign gesture into a file and recall later for comparsion against the signers gesture.

    2. I have found kinecttoolbox.codeplex.com project looks promising but if anyone has anything that they think is better suited for dynamic gestures please share.

    3.  Actually compare the gesture data file loaded into the program or load gesture files one at a time until the current sign is recoginized against what the person is currently signing.

    I have the project page here if anyone wants to help directly to the project:

    http://kinectsignlanguage.codeplex.com


    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - "Sherlock holmes" "speak softly and carry a big stick" - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda




    Monday, March 12, 2012 8:20 PM

Answers

All replies

  • If anyone has any ideas you can contribute them. Also, anyone wanting to help with the project once I figure out how to record gestures to file and load them back in my program and compare them then i need someone to help me record sign language signs and then I will make a database out of it that the program will access. but for now i need to make a recorder program that can load the gesture back into it and then recoginize the data from the file as a gesture and compare against the current one loaded into ram from current active user.


    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - "Sherlock holmes" "speak softly and carry a big stick" - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda

    Monday, March 12, 2012 11:19 PM
  • We have just announced that the 1.5 release will include Kinect studio, which will (among other things) allow you to record and replay sensor data from the kinect.  This should help in the effort you're describing.  I'm excited to see what yoy come up with in the ASL space. :)  

    (Blog post for reference: http://blogs.msdn.com/b/kinectforwindows/archive/2012/03/26/what-s-ahead-a-sneak-peek.aspx)

    • Marked as answer by The Thinker Wednesday, March 28, 2012 2:52 PM
    Wednesday, March 28, 2012 7:00 AM
  • Does that include gesture data loading from and too file since gesture support is supposedly at least partially going to be supported from what i've got on feedback from previous posts? I would love to do like kinect toolbox and just save gesture to file and then load it back so i can do this so if gesture support is their wheter partial or full i will be excited. Thanks for your help.

    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - "Sherlock holmes" "speak softly and carry a big stick" - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda



    • Edited by The Thinker Thursday, March 29, 2012 1:44 PM
    Wednesday, March 28, 2012 2:56 PM