OpenCV and Kinect SDK RRS feed

  • Question

  • Hi I am not sure if this was answered before, I went through the other threads about this, but I am a beginner and I don't quite understand them :(

    I want to grab frames of RGB and depth data from the kinect and use openCV to process them to detect and recognize people. In openCV, to grab a frame from a camera connected to my computer, I would do use cvCreateCameraCapture() to get a cvCapture pointer from which I can grab frames.

    My questions are:

    - I see it is not so simple when your camera is a kinect. How can I take RGB and depth data from the kinect and put them into images that I can process in OpenCV?

    -How do I know what depth data corresponds to what RGB data. I know this is a calibration issue?

    -I was also told that theKinect depth sensor gives a one channel 11-bit image that contains depth data. Does openCV have funtions that can process 11 bit images? If not how do I convert the 11 bit images to something that I can pass to opencv functions?

    -Can I use the C# version of the sdk? or do I have to use the C++ one? Do I have a choice?



    Sunday, November 27, 2011 11:29 PM

All replies

  • I don't know about openCV, you might want to visit their site for information like that.  In case you weren't aware, there is Skeletal Tracking built into the Kinect for Windows platform, you might look into whether it would work for you.

    I presume openCV has an input data format.  I would imagine that you would need to handle each frame of Kinect input, and transform that data into a frame of openCV input.  Same for the 12 bit depth data.

    For correlating RGB and Depth data, there is a function to help you. The C#/VB one is NuiCamera.GetColorPixelCoordinatesFromDepthPixel() , there is a similar C++ API. 

    You choice of C# or C++ is up to you... C#/WPF is easy to get started with and very powerful, but if openCV is C++ only then C++ might be better.

    Tuesday, November 29, 2011 1:24 AM
  • Hi,

    Here are some hints to get you started : 

    You should initialize your kinect through Microsoft NUI API

    Then you can easily grab frame from your kinect Those frame will be NUI_IMAGE_FRAME

    From those you extract the  frame texture NuiImageBuffer (that are your pixels)

    You copy them in a cv::Mat or IplImage regarding what you want to use

    Then you use your openCV routines to process this image.


    Good luck


    • Proposed as answer by ykbharat Sunday, May 6, 2012 12:20 PM
    Wednesday, November 30, 2011 7:39 PM