Kinect point clouds, OpenGL and banding RRS feed

  • General discussion

  • I'm reading a depth stream from the Kinect SDK and displaying 3D point clouds using vertex shaders in OpenGL. Everything seems to work properly, but the data is heavily banded, as in the linked pic.

    Banding image

    All the pictures of other projects I see online show really beautiful, detailed point clouds. Is there a pre-processing step I'm missing? I've tried shifting the depth value left by 3 bits(depth << 3) which gives the results shown - without the shift, the banding is even worse.

    More details and full, annotated source for the example are available at this Github project

    Saturday, July 19, 2014 7:17 PM

All replies

  • Are you using v1 or v2?

    In v1, there are 2 two types of depth frame formats, one that has the 16bit values for depth and 16bits for player and the other is the compressed version with playermask part of the 16bits. So, it might be you are passing the 4byte(32bits) version to your shader. This would account for half the data missing.

    Additionally, you may want to ensure you are mapping the depth frame to camera space(world/skeleton) instead of the depth frame itself. the depth frame is the projected space mapped onto the plane.

    Carmine Sirignano - MSFT

    Monday, July 21, 2014 8:47 PM
  • Hi Carmine, thanks for the reply.

    I'm using Kinect v1, the Kinect for Windows version with SDK 1.8. I'm initializing Nui with NUI_INITIALIZE_FLAG_USES_DEPTH_AND_PLAYER_INDEX and opening an image stream with NUI_IMAGE_TYPE_DEPTH_AND_PLAYER_INDEX, then iterating over the array taking only the "depth" field from the NUI_DEPTH_IMAGE_PIXEL array returned by NuiImageStreamGetNextFrame, discarding "playerIndex".

    As for the frame mapping, are you referring to the function NuiTransformDepthImageToSkeleton? Can you tell me the details of what that function is doing? It makes much more sense for me to apply that operation in vertex shader as I don't even store the vertex positions anywhere on the CPU/RAM side(only the depth image before it is uploaded to GPU). It's a perfect fit for GPU work if it is a simple linear transform.

    Even with a linear transform applied, I can't imagine the banding in the linked image smoothing out noticeably. Would you agree that the data in that image looks incorrect? It looks like I'm seeing the contents of the entire depth image in the point cloud, I don't think any pixels are missing. The gross shapes of the point clouds are approximately true to form. What is the smallest depth discrepancy the Kinect should be able to discern at a range of 1 meter?
    Wednesday, July 23, 2014 2:14 AM