Kinect idea. Come into my livingroom! RRS feed

  • General discussion

  • Hi everybody,

    I have a question about the possibilities of the Kinect camera streams. I have seen seperate projects that allows the depth image to be sent like a webcam video stream. What I would like to do is combine two depth (and color) video streams and combine them into one. And clip all but the person from one stream. That way it will seem like that person is standing in the other persons livingroom :-)  Depending on quality and speed all kinds of things will be possible...

    What I'm looking for is some clues on finding what is nessacary to make this.

    Thanks in advance,

    Dave, NL

    Saturday, March 10, 2012 1:25 PM

All replies

  • This is definitely something that can be done and you touched on the 2 key points: Quality and Speed. Below are my thoughts on how you might proceed and by no mean the only way.

    Are you trying to do this real-time over a network or would you want to save a snapshot of other living rooms data?

    Take a remote scenario; you will have to know all of the networking data and communication to get the depth/color from the remote location. The amount of data for one uncompressed RGB color image is ~1MB(640x480x4bbp) not adding in depth. Will you have to removing a person from the remote living room data? If so, you will have missing color information where they were standing. Having to recreate this will add more overhead on getting this done in such a short amount of time for video.

    That aside, mapping the local user into the other color image can be done in a number of ways. The standard way is a Chroma key style removal (color only). The area around the person in the image data would be set to an alpha value of 0. Another idea would be removing the background and some foreground based on the depth data. This discussion might prove helpful on this technique:

    Placing that person "into" the living room color image from the remote living room can be a bit tricky. Depending on the level of "realism" you are looking for, you could just display the 2 color bitmaps on top of each other (WPF 2 <image>)in the UI. If you are trying for realism, for example there is a coffee table and you wanted them to appear between the coffee table and the sofa, then you will have to analyze the remote depth data and determine where the pixels are for the table, and remove those from the person image as well.

    Tuesday, March 13, 2012 8:41 PM
  • Scott has posted his code for his kinectcam rgb feed so you could modify it and repost it with your modifications in your own project:

    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - "Sherlock holmes" "speak softly and carry a big stick" - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda

    • Edited by The Thinker Thursday, March 15, 2012 7:15 PM
    Thursday, March 15, 2012 7:14 PM