Windows universal Kinect floor plane tilt and height RRS feed

  • Question

  • Using UWP code I need a way to convert the Kinect joint positions from Camera space to World Space (Alter the Vector3 positions so they are not measured from the Camera position but up from the floor plane and adjusted for the camera tilt)
    This was achievable with the Kinect Preview Library before by using the Vector4 BodyFrame.FloorClipPlane
    It was easy to get the Camera height from the floor and the tilt angle on the sensor.

    I've looked in to the newer Windows Universal Samples that use the Windows.Media.Capture.Frames APIs
    and I can access the skeletal tracking using custom media stream and could do conversions to 2D image spaces but it would appear the simple camera height and tilt access did not get ported across to these new APIs.

    Can anyone confirm? Thank you.

    More details:
    I do see with UWP sample code a PoseTrackingEntity may have CustomData.. and perhaps the Camera orientation / floorPlane data is but what format that might be is not documented.
    I've also looked over SpatialCoordinateSystem & CameraIntrinsics & DepthCorrelatedCoordinateMapper classes but I don't think they provide me with Camera Height and Tilt.
    Finally HoloLens seems to have a concept of a 3D position for itself but it is not clear if Kinect would provide similar properties
    Monday, January 14, 2019 2:56 PM


  • So I found the floor plane Vector4 in the CustomData and so can now get height and tilt.
    Really this should be documented somewhere as the boilerplate comment that GetCustomData "Retrieves the location of Provider specific entity data in the frame buffer" does not help much if the Provider "Microsoft" in this case has not listed this info anywhere.

    Also of note there are 68 bytes of custom data with each PoseTrackingEntity (Body in old SDK terms) that is not described. Possibly it's the hand status (open or closed and some other data, Face orientation perhaps ? ).

    Anyway back to height and tilt... so if you are using the UWP sample that has the PoseTrackingFrame class
    Then in the C++ version it has a BufferView^ GetCustomData(); Something similar must be there in C# version.

    The CustomData has a size of 16 bytes. If you get a pointer to the start of that data you can cast to a float4. Each float being 4 bytes. That's the floor plane.

    BufferHelper helper(poseTrackingFrameCustomData->Buffer);
    auto plane = helper.GetPtrAt<Windows::Foundation::Numerics::float4>(poseTrackingFrameCustomData->Offset);
    char planeString[256]; // just for my debugging
    printf_s(planeString, "Our plane is %.2f %.2f %.2f %.2f \n", plane->x, plane->y, plane->z, plane->w);

    plane->w is the height of the camera in meters. So Expect something like 0.7
    and I believe tilt in degrees would be:

     Math.Atan(plane->z / plane->y) * (180.0 / Math.PI);

    • Marked as answer by davidkitman Wednesday, January 16, 2019 3:43 PM
    Wednesday, January 16, 2019 3:15 PM