none
AirGuitar Kinect Project Guidance RRS feed

  • General discussion

  • Hi all,

    I am a student planning on making an AirGuitar (or possibly AirDrum) modification/hack for the Kinect. This project is classified as a Personal Project, which is part of the MYP (IB) curriculum and is very important to me. I roughly have 3 to 4.5 months to finish the project. Right now I have many questions on the feasibility and the difficulty of the project. I have completed AP Computer Science A and know some knowledge about Java and a little about C, I think I am rather good with algorithms and not so good on the actual code logic.

    Here are my questions:

    1. Is the allotted time frame enough for me to finish the project, in other words do you think that you can finish it within 4 months? ( I have a minimum of an hour a day to work on this)

    2. What operating system is the most suitable for this project? I have Macs with Bootcamp (Windows 7) installed on them.

    3. What hardware/software will I need for this project? (Obviously I need a Kinect and the Kinect SDK)

    4. How hard would this be? (What skills will I need, and if I need to study on some things)



    I deeply appreciate the fact that you guys are giving up your precious time to help me with this project. Your help will not be unnoticed and is probably crucial to my success. Anybody who has helped in anyway will without a doubt be credited and thanked.



    Thanks in advance,



    Harry
    Wednesday, October 19, 2011 1:34 PM

All replies

  • Harry, I'll try to answer your questions the best I can, but some will require additional clarification.

    1) I think air drums will be a lot easier to implement than air guitar, because for coarse-level movements used when playing drums you will be able to leverage the full-body skeletal tracking API provided by Kinect SDK, which gives you a stream of 20 joint positions (e.g.: right hand, right elbow, left knee, right foot, head, spine, etc). For air guitar you would have to write your own finger tracking code on top of the raw depth image stream provided by kinect, and that would be a much much more challenging project. Unless you already have a really good knowledge base in image processing techniques, just recognizing fingers accurately enough for your needs (even without adding the code for the rest of your application UI) could take you months. That being said, there are some people writing code for this purpose (e.g.: http://social.msdn.microsoft.com/Forums/da-dk/kinectsdk/thread/dd4d3e9a-fa8f-4ff7-9262-d9e8caa5952a), and you might be able to leverage some of it.

    The problem of writing an application to control an air guitar or air drum does seem feasible in 4 months, if you scope it down enough. Make sure you think hard about what is the bare bones minimally functional product you would like to do and try to write that in 1-2 months (with plenty of time to spare), then add some interesting feature every 2 weeks (or less).

    2) The Kinect SDK Beta has only been tested in machines with windows 7 installed. Some people have had luck with using SDK in windows virtual machines, but this is not an officially supported scenario and a lot of people have had problems. So, I would recommend not relying on a Mac for this.

    3) The readme (http://research.microsoft.com/en-us/um/redmond/projects/kinectsdk/docs/readme.htm) lists the HW/SW requirements

    4) Well, it depends on how rich you want to make the interaction of your users with your application. If you list out your minimal requirements for this project, it would be easier to answer those kinds of questions.

    Eddy


    I'm here to help
    Friday, October 21, 2011 2:39 AM
  • Hey Eddy,

     

    First of all thanks for your comprehensive answers. :D

     

    Okay, the original idea for my AirGuitar project was that the code first finds the person using the given full-body skeletal tracking API, then adds a virtual guitar to the person which would be displayed on the camera's output. The neck of the guitar would be like a ruler, where the player slides his/her hands (wrist) along the ruler. And depending on how far away it is from the center of the body, the pitch of the notes would be higher or lower. 

    On the 'ruler' there would be letters indicating the chord or note that is going to be played. First crude effort diagram:

    tip of the guitar-> {A | B | C | D | E | F | G|*    guitar body  *}

     

    So the person's hand or wrist can be in the area of each letter. This significantly makes the project easier since the target audience was for someone that didn't know how to play guitar (so they wouldn't know the proper fingering for chords anyway) and wanted to experiment the feel of strumming a guitar.

     

    Harry

    Friday, October 21, 2011 9:29 AM
  • Oh, ok, that sounds a lot simpler :)

    So, you basically have the following sub-problems:

    1) From skeleton joint positions, find proper guitar position and alignment
    2) From guitar position and alignment, designate A,B,C,etc. note regions
    3) Display guitar with proper note annotations, together with user, in some format
    4) Determine which note, if any, is being indicated by user's hand/arm position.
    5) Detect guitar strumming gesture in other user's arm
    6) output correct guitar sound for note

    So, I'd suggest simplifying problem by starting out fully in 2D space, only using X and Y skeleton joint information, and rendering stick figures for both user and guitar. Also, just have 7 pre-recorded sounds for a guitar playing A, B, C, D, E, F, G in a down stroke (you can add up-stroke later).

    This should be totally feasible in a month or two, spending one to a couple of hours each day, and then you can do improvements one at a time, such as supporting both right and left handed stance, detecting more strumming patterns, placing guitar in 3D space, or even super-imposing a nice looking guitar 3D model in a natural looking pose on top of user's RGB image, but I think you should try to do these after you have the basic application working.

    Eddy


    I'm here to help
    Friday, October 21, 2011 6:39 PM
  • Hello guys!

    After four months of hard work, I have finally finished my air guitar! It didn't turn out to be as complex as the one described so well by Eddy but nonetheless its an Air Guitar! I has all the chords of the C Major key. A minor, B diminished, C major, D minor, E minor, F major, G major

    Feel free to play around with my program:

    Download source from Skydrive:

    https://skydrive.live.com/redir.aspx?cid=35f120abd4dcf8c0&resid=35F120ABD4DCF8C0!139&parid=root

    System Requirements

    Windows 7 32bit

    Kinect SDK BETA 1 IS HIGHLY RECOMMENDED. EDITS TO THE CODE WILL MAKE IT POSSIBLE TO BE RUN WITH BETA 2 but I don't know how :P

    Visual Studio

    2.66ghz Core 2 Duo

    4gb RAM

    500mb Free space

     

    *** After opening .sln in visual studio, make sure to install the Coding4Fun Kinect Toolkit, install the references within visual studio***

    Link -->    http://c4fkinect.codeplex.com/releases/view/76271

     

    Feel free to edit, share whatever. Just credit the right people if you want to share - in this case Ray Chambers and Harry Li.

     

    Enjoy,

    (if you have any questions feel free to email me @ harry_de_best@hotmail.com or Skype username: harry_de_best

    Harry

     

     

     

    Tuesday, January 31, 2012 6:03 PM