none
depthmap has black/null values on left edge, does it affect NuiTransformDepthImageToSkeleton? RRS feed

  • Question

  • Hello. I have noticed with my Kinect, in my depthmaps, in my code, and in the Toolkit Kinect Explorer, that there are several columns of null data. In my case, the first 4 columns of a 320x240 depthmap are always 0.

    Is there any shift of data associated with this? Specifically, is the center of the data actually not at 160 but is instead at 162 or 164?

    As I look at the NuiTransformDepthImageToSkeleton() code, I don't see anything that would take into account a shift of data. I can't see the code of the newer MapDepthPointToSkeletonPoint().

    I asked this question because as I do some mapping and coordinate set conversations, I am noticing my data tends to be off in the X direction. And I'm wondering if this is related to these null columns. And if there is a way I can compensate for them.


    --Dale

    Friday, May 17, 2013 4:55 PM

All replies

  • The skeleton positions are derived directly from the depth data in exactly the form that you see it. So no adjustment should be necessary to compensate for this region of empty depth data.


    John | Kinect for Windows development team

    Monday, May 20, 2013 8:25 PM
  • So that we are mutually clear, I'm not speaking of "skeleton position" as you write.

    Instead, I am inquiring about skeletal coordinates using that specific API. The API is just a macro so the code is exposed (a good thing). There is no code there which adjusts for shift off center.

    This lack of visible code shifting and my real-world experience in measuring this shift with a measuring tape has me interested in leaning more. I have the accelerometer rotating data with excellent results. Now, I see the X shift.  Is this region of empty depth data the result of some deeper APi/hardware shifting (like the factory adjustments to align)? And therefore after that alignment higher data exposed (e.g. depthmap) has shifted data so that APIs and macros will correctly gen their results?

    Techy inquiring minds would like to learn rather than guess-trial-error.


    --Dale

    Tuesday, May 21, 2013 1:17 AM
  • Would you allow a run-on? In addition to confirming/clarifying position vs. coordinates...

    The accelerometer...it is aligned to the depth camera as the depth camera is aligned to the rgb camera? I've written code to rotate all the data. When I drag my cursor across the floor I see in my depth (or rgb) output -> mapped to skel coord -> rotates by accel => the results are showing the Y value (distance from floor) varying as the distance increases from the Kinect. The algorithms suggest that the floor is lower when closer (Z) to the Kinect and higher away from the Kinect. I'm seeing a variation of about 8cm in height difference front to 4m back. If the accelerometer wasn't aligned with the depth camera, then this could explain the results.


    --Dale

    Thursday, May 23, 2013 2:48 PM
  • I can partially answer my own question. Referencing http://msdn.microsoft.com/en-us/library/jj663790.aspx

    • The accelerometer has a 1 degree accuracy. Given this, at 4m, the Y variation can already be 7cm.
    • In addition, there is a temperature drift that occurs which can be up to +/3 degrees. That could introduce up to 20cm of Y variation at 4m.

    Worst case, even if the accelerometer was factory aligned to the depth sensor, the accelerometer could vary a Y reading at 4m up to 27cm from a reading taken directly after a cold start.

    Given all that, knowing about factory alignment is moot.

    I would appreciate the clarify/confirmation on the position/coordinate inquiry. :-)


    --Dale

    Thursday, May 23, 2013 7:46 PM
  • Got it. Is the X shift uniform at a given depth (same amount of error, regardless of position with in the field of view)? I'm trying to get a handle on the nature of the problem you're observing.


    John | Kinect for Windows development team

    Monday, June 3, 2013 5:41 PM
  • Here's what I can share. I use a carpenters square to align the external face of a Kinect perpendicular to the planks of wood on my floor (which are likely straight). I acknowledge that such alignment to the external plastic housing could be a source of error as is my carpenters square approach. ;-)

    I put a block of wood at several distances from the Kinect on the floor and aligned its edge with the straight cut of the wood. Therefore the wood's edge is at coordinate X=0.0.

    I then have code which captures the depthmap and shows me a rendering of it. I can then use my mouse to move over the map and use the pixel coords to then call NuiTransformDepthImageToSkeleton(). I am capturing a 640x480 depthmap.

    At z=1.007m, x=-0.024
    At z=1.536, x=-0.021
    At z=2.069, x=-0.025
    At z=2.599, x=-0.027

    I will attach two pictures. These are screenshots of a 320x240 depthmap. One is from the Kinect Explorer as part of the toolkit. Notice on the left of the depthmap the columns of black data. The second is from a Max patch within which my C code calling your APIs is running. It shows an equivalent depthmap + the data itself. Notice the first 4 depthpixels of each row are zero. This matches the black/null data seen in both apps. I have two Kinects and this 4 columns of null repro's on both.

    Is there a relation between this null data on every row and the X offset that I am experiencing?


    --Dale

    Tuesday, June 4, 2013 5:57 PM
  • To the best of my knowledge, those blank columns of pixels do not represent a shift in the data, but are simply an area where no data has been captured.

    I notice that each of your X measurements is off by 21 to 27 mm. Are you certain that you are aligning your scene with the depth camera and not the color camera? They are about 25 mm apart.


    John | Kinect for Windows development team

    Thursday, June 6, 2013 5:52 PM