none
Weird behaviour of coordinate mapping RRS feed

  • Question

  • Hi,

    I'm working on a project that requires colored point cloud data, therefore I have to perform depth-to-color mapping. Now the mapping is kind of weird and I am wondering why that is and what I can do about it. According to the CoordinateMappingBasics sample, I coded this:

    #define KINECT_DEPTH_X_RES     512

    #define KINECT_DEPTH_Y_RES     424

    #define KINECT_DEPTH_PIXELS    ( KINECT_DEPTH_X_RES * KINECT_DEPTH_Y_RES )

    #define KINECT_COLOR_X_RES     1920

    #define KINECT_COLOR_Y_RES     1080

    #define KINECT_COLOR_PIXELS    ( KINECT_COLOR_X_RES * KINECT_COLOR_Y_RES )

    //in class definition:

    typedef unsigned int   NIColorPixel;

    typedef unsigned short NIDepthPixel;

    std::vector<NIColorPixel>      fullColorBuffer;

    std::vector<NIColorPixel>      outColorBuffer;

    std::vector<NIDepthPixel>      depthBuffer;

    //in constructor:

    this->depthBuffer.resize( KINECT_DEPTH_PIXELS );

    this->outColorBuffer.resize( KINECT_DEPTH_PIXELS );

    this->fullColorBuffer.resize( KINECT_COLOR_PIXELS );

    this->colorCoordinates.resize( KINECT_DEPTH_PIXELS );

    this->fullColorResX = KINECT_COLOR_X_RES;

    this->fullColorResY = KINECT_COLOR_Y_RES;

    //in update loop:

    hr = this->coordinateMapper->MapDepthFrameToColorSpace( this->depthBuffer.size(), (UINT16*) &this->depthBuffer[0], this->depthBuffer.size(), &this->colorCoordinates[0] );

    if( SUCCEEDED( hr ) )

    {

            auto itOutCol = this->outColorBuffer.begin();

            auto itColCoord = this->colorCoordinates.begin();

            for( ; itOutCol != this->outColorBuffer.end(); ++itOutCol, ++itColCoord )

            {

                   NIColorPixel col = 0;

                   if( itColCoord->X != -std::numeric_limits<float>::infinity() && itColCoord->Y != -std::numeric_limits<float>::infinity() )

                   {

                           int fullColX = static_cast<int>( itColCoord->X + 0.5f );

                           int fullColY = static_cast<int>( itColCoord->Y + 0.5f );

                           if( ( fullColX >= 0 && fullColX < this->fullColorResX ) && ( fullColY >= 0 && fullColY < this->fullColorResY ) )

                                   col = this->fullColorBuffer[fullColX + fullColY * this->fullColorResX];

                   }

                   *itOutCol = col;

            }

    }

    Now, when I display the color data, I get weird double-areas on the right hand side of depth edges:


    I'm using SDK version 2.0.1410.19000 from 2014 (which is the most recent one, right?).


    Friday, August 26, 2016 2:31 PM

All replies

  • I've seen the same behaviour and I believe it is fundamentally because the depth and RGB cameras are sideways separated and have different field of view. You can do the projection also the other way around (map RGB to depth) but projector shadowing is then much more apparent. So you either get double RGB pixels or larger areas of completely black (depth unknown) pixels.

    Tuesday, August 30, 2016 12:20 PM