none
How to get the top of the head RRS feed

  • Question

  • Hi, 

    For my internship, I have to code in C++ for a kinect for xbox360 and I must get the position of the top of the head ( indeed, as I see it, kinect spot the middle of the head). 

    My idea was to display the image of the user, then just find the pixel the most " higher " on the screen and then, convert it on " real life " coordinates. But it won't work if the user put his hands up.

    I use the SkeletonBasics-D2D sample and I have read this : http://msdn.microsoft.com/en-us/library/nuisensor.inuicoordinatemapper.mapdepthframetoskeletonframe.aspx but I don't think it is very helpful for my problem. ( I can't give links)

    "Maps the entire depth frame to skeleton space. Any depth pixel without a corresponding skeleton point will result in a sentinel value." Does this mean that only the pixels on the skeleton will be ok ? Or can I use it for calcul the top head position ?


    Do you have any idea ? maybe someone already did it and can explain things / give advice ?

    ps: My tutor wants something very accurate so I can't just add 10cm to the skeleton head position

    I would like to apologize if there is any mistake, I don't speak english very good

    Thanks, 
    Dalkeri

    Monday, May 6, 2013 11:56 AM

Answers

  • One possible approach:

    1. Start with the head position as reported by the skeleton.
    2. Use INuiCoordinateMapper::MapSkeletonPointToDepthPoint to get the corresponding coordinates in the depth image.
    3. Look up the playerIndex for those coordinates in the depth image.
    4. Starting from those coordinates, begin examining pixels in the depth image that are "above" the coordinates obtained in step 2 (that is, decrement the Y coordinate).
    5. The last such pixel with a playerIndex that matches the one you found in step 3 will be the top of the head. Take the coordinates and depth value for this pixel, and use INuiCoordinateMapper::MapDepthPointToSkeletonPoint to convert them to coordinates in the same space as the skeleton.

    Note that if the person has their hand or arm resting directly on top of their head, the answer won't be correct, but as long as there is nothing directly on top of the head, it should be fine.


    John | Kinect for Windows development team

    • Marked as answer by Dalkeri Tuesday, May 7, 2013 5:49 AM
    Monday, May 6, 2013 5:50 PM

All replies

  • One possible approach:

    1. Start with the head position as reported by the skeleton.
    2. Use INuiCoordinateMapper::MapSkeletonPointToDepthPoint to get the corresponding coordinates in the depth image.
    3. Look up the playerIndex for those coordinates in the depth image.
    4. Starting from those coordinates, begin examining pixels in the depth image that are "above" the coordinates obtained in step 2 (that is, decrement the Y coordinate).
    5. The last such pixel with a playerIndex that matches the one you found in step 3 will be the top of the head. Take the coordinates and depth value for this pixel, and use INuiCoordinateMapper::MapDepthPointToSkeletonPoint to convert them to coordinates in the same space as the skeleton.

    Note that if the person has their hand or arm resting directly on top of their head, the answer won't be correct, but as long as there is nothing directly on top of the head, it should be fine.


    John | Kinect for Windows development team

    • Marked as answer by Dalkeri Tuesday, May 7, 2013 5:49 AM
    Monday, May 6, 2013 5:50 PM
  • Hi Dalkeri,

    a simple solution:
    starting from the pixel coordinate of the HeadJoint and it's depth value in the depthFrame, with a simple loop, you can check the depth value of the pixel located in "the upper position".
    If the depth value is +5cm (value to be tuned) respect the starting position you know the top of the head else you need to check a pixel in upper position.

    a different solution:
    study OpenCV ;)

    Good job
    Monday, May 6, 2013 8:20 PM
    Moderator
  • In the beginning of my internship, I downloaded OpenCV because I wanted to try a sample in the developer tooltkit browser but my tutor said I don't need this. So I guess, I have to try your first solution.

    Well thanks you for your answers,I try this as soon as possible and I'll comme back here if there is any mistake ( but it shouldn't )

    • Edited by Dalkeri Tuesday, May 7, 2013 5:49 AM
    Tuesday, May 7, 2013 5:47 AM
  • I don't understand how I am suppose to use INuiCoordinateMapper::MapSkeletonPointToDepthPoint, I have always errors.

    I try like this :

    INuiCoordinateMapper map;

    map.MapSkeletonPointToDepthPoint(coordTetePointeur, NUI_IMAGE_RESOLUTION_320x240 , coordTeteDepth);

    the error is on the underlined map. It speaks about pure virtual fonction but I have no idea how to resolve it. 

    I've read that I must have a class with a legacy with INuiCoordinateMapper but I have nothing to put in... I have nothing to put in the constructor ...

    Do I have to create a new class ?

    thanks for your help and advices,

    Dalkeri


    • Edited by Dalkeri Tuesday, May 7, 2013 8:57 AM
    Tuesday, May 7, 2013 8:30 AM
  • You get the coordinate mapper from the sensor:

    INuiCoordinateMapper* pMap;
    pNuiSensor->NuiGetCoordinateMapper(&pMap);
    pMap->MapSkeletonPointToDepthPoint(...);
    pMap->Release();


    John | Kinect for Windows development team


    Wednesday, May 8, 2013 1:21 AM
  • Thanks again.

    I must put it in CreateFirstConnected() ? ( it is where it counts the sensors and initialize them ) Or can I put it where I want ? ( and Release(); at the end of the program)

    I am sorry, I know I'm bad. But I don't have kinect at home, and I don't work until monday.

    Wednesday, May 8, 2013 10:21 PM