none
How get the the joint x,y,z coordinates in C++ RRS feed

  • Question

  • Hey guys,

    since i am a "free time" programmer i am fighting to write a running program to get the joint coordinates using the SDK in C++.

    It would be great is someone could help me.  In my case the initialization and the skeleton recognition works, except the coordinate output.

     

    thanks for any help in advance

     

    Thursday, June 30, 2011 12:31 PM

Answers

  • Eddy,  i finally solved the problem. The key was the NUI_SKELETON_COUNT function. Now the program works fine with fast player detection. :)

    For any interested i ll put the code, without any warranties. Parts of it came from post of Bogdans( thank!)

    Hope someone can use it.

     

    #include <windows.h>
    #include  "MSR_NuiApi.h"
    #include <string>
    #include "MSRKinectAudio.h"
    #include "MSR_NuiImageCamera.h"
    #include "MSR_NuiProps.h"
    #include "MSR_NuiSkeleton.h"
    #include "MSRKinectAudio.h"
    #include "NuiImageBuffer.h"
    #include "resource.h"
    #include <mmsystem.h>
    #include "SkeletalViewer.h"
    int x;
    int null = 0;
    int m_eins = 1;
    HRESULT hr = S_OK;
    int nCount = 0;
       

    HANDLE        m_hNextDepthFrameEvent;
        HANDLE        m_hNextVideoFrameEvent;
        HANDLE        m_hNextSkeletonEvent;
        HANDLE        m_pDepthStreamHandle;
        HANDLE        m_pVideoStreamHandle;



    int connect()
    {
        hr = NuiInitialize( NUI_INITIALIZE_FLAG_USES_SKELETON);
      if( FAILED( hr ) )
      {
        printf ("nicht initialisiert!! KINECT verbunden? \n");
        system("pause");
      
      }
      else
      {
        // m_hNextSkeletonEvent = NULL;
      //m_hNextSkeletonEvent = CreateEvent( NULL, TRUE, FALSE, NULL );
     
        NUI_SKELETON_FRAME SkeletonFrame;
      NuiSkeletonTrackingEnable;

      //NuiSkeletonTrackingEnable( m_hNextSkeletonEvent, 0 );
      }
         
     
    return 0;
    }



    int main()
    {
        connect();
       
        NUI_SKELETON_FRAME SkeletonFrame;
    //hr = NuiSkeletonGetNextFrame( 0, &SkeletonFrame );
      while (true)
        {
            NuiSkeletonGetNextFrame( 0, &SkeletonFrame );
           

        bool bFoundSkeleton = true;
        for( int i = 0 ; i < NUI_SKELETON_COUNT ; i++ )//////////////////hier werden mehrere Skeletons abgefragt!!!!!!!!!!!!!!!
        {
           
            ////////////////////////hier wird nur ein skeleton abgefragt!!!!!!!
            if( SkeletonFrame.SkeletonData[i].eTrackingState == NUI_SKELETON_TRACKED )////////////////prüft ob skeletondaten verfügbar sind
            {
                //bFoundSkeleton = false;
               
            //printf(" Found skeleton");

       
        float hand_l_x=SkeletonFrame.SkeletonData[i].SkeletonPositions[NUI_SKELETON_POSITION_HAND_LEFT].x;
        float hand_l_y=SkeletonFrame.SkeletonData[i].SkeletonPositions[NUI_SKELETON_POSITION_HAND_LEFT].y;
        float hand_l_z=SkeletonFrame.SkeletonData[i].SkeletonPositions[NUI_SKELETON_POSITION_HAND_LEFT].z;   
               
       
            //right hand
        float hand_r_x=SkeletonFrame.SkeletonData[i].SkeletonPositions[NUI_SKELETON_POSITION_HAND_RIGHT].x;
        float hand_r_y=SkeletonFrame.SkeletonData[i].SkeletonPositions[NUI_SKELETON_POSITION_HAND_RIGHT].y;
        float hand_r_z=SkeletonFrame.SkeletonData[i].SkeletonPositions[NUI_SKELETON_POSITION_HAND_RIGHT].z;
        //if (hand_l_x!=0 && hand_l_y!=0)
        if (hand_l_x!=0 && hand_r_z!=0 )
        {
            //printf("HandLeft X=%4.2f    ",hand_l_x,"HandLeft Y=%4.2f    ",hand_l_y,"HandLeft Z=%4.2f    \r\n",hand_l_z);
            //printf("HandRight X=%4.2f    ",hand_r_x,"HandRight Y=%4.2f    ",hand_r_y,"HandRight Z=%4.2f    \r\n",hand_r_z);   
            printf("HandLeft X=%4.2f    ",hand_l_x,"HandLeft Y=%4.2f    ",hand_l_y,"HandLeft Z=%4.2f    \r",hand_l_z);
            printf("HandRight X=%4.2f    ",hand_r_x,"HandRight Y=%4.2f    ",hand_r_y,"HandRight Z=%4.2f    \r",hand_r_z);   

        }

        //////////////////////////////////////////////////////////////       
        else
                    {
                        
                NuiSkeletonGetNextFrame( 0, &SkeletonFrame );
                    }
           
            }       
        }

        // no skeletons!
        //
        if( bFoundSkeleton )
        {
           //    printf("NOT FOUND");
            //system("pause");
            //return 0;
        }

        // smooth out the skeleton data
        NuiTransformSmooth(&SkeletonFrame,NULL);
        // we found a skeleton, re-start the timer
       //NuiSkeletonTrackingEnable;

        }
     

                           

                         

    system ("pause");                       
    return 0;
    }

     

     

    • Marked as answer by ivan_balvan Monday, July 4, 2011 12:06 PM
    Monday, July 4, 2011 12:06 PM

All replies

  • Ivan,

    These coordinates are available as part of NUI_SKELETON_DATA.SkeletonPositions array, which you receive as part of a NUI_SKELETON_FRAME. Have you looked at SkeletalViewer C++ sample? It should have installed in your machine under C:\Users\Public\Documents\Microsoft Research KinectSDK Samples\NUI\SkeletalViewer\CPP

    Hope this helps,
    Eddy


    I'm here to help
    Friday, July 1, 2011 2:47 AM
  • Eddy thanks for your answer. You put you finger in the wound. Actually i read the example but did not get the point how exactly use NUI_SKELETON_DATA and NUI_SKELETON_FRAME functions. But in the mean time i tried this and that and. Additionally inspired by Bogdans code in the post :"My kinect don't detect my skeleton"  i was able copy and pasted one working version.

    But now another problem arised. This program is very unpredictable in the meaning, it takes a lot of time to recognize the person.It seems the recognition itself happens randomly.

    Here is the code and please dont comment the esthetics of it. I know its not perfect :)

    Would be great if someone can help me in the recognition problem.

    Any help is welcome.

     

    #include <windows.h>
    #include  "MSR_NuiApi.h"
    #include <string>
    #include "MSRKinectAudio.h"
    #include "MSR_NuiImageCamera.h"
    #include "MSR_NuiProps.h"
    #include "MSR_NuiSkeleton.h"
    #include "MSRKinectAudio.h"
    #include "NuiImageBuffer.h"
    #include "resource.h"
    #include <mmsystem.h>
    #include "SkeletalViewer.h"
    int x;
    int null = 0;
    int m_eins = 1;
    HRESULT hr = S_OK;
    int nCount = 0;
       

    int connect()
    {
        hr = NuiInitialize( NUI_INITIALIZE_FLAG_USES_SKELETON);
      if( FAILED( hr ) )
      {
        printf ("nicht initialisiert!! KINECT verbunden? \n");
        system("pause");
        return hr;
      }

         
     
    return hr;
    }



    int main()
    {
        connect();

     
      NUI_SKELETON_FRAME SkeletonFrame;
      NuiSkeletonTrackingEnable;

                       

      while (true)
        {
            NuiSkeletonGetNextFrame( 0, &SkeletonFrame );
           

       
        //for( int i = 0 ; i < NUI_SKELETON_COUNT ; i++ )//////////////////hier werden mehrere Skeletons abgefragt!!!!!!!!!!!!!!!
        //{
           
            ////////////////////////hier wird nur ein skeleton abgefragt!!!!!!!
            if( SkeletonFrame.SkeletonData[1].eTrackingState == NUI_SKELETON_TRACKED )////////////////prüft ob skeletondaten verfügbar sind
            {
               
               
            //printf(" Found skeleton");

       
        float hand_l_x=SkeletonFrame.SkeletonData[1].SkeletonPositions[NUI_SKELETON_POSITION_HAND_LEFT].x;
        float hand_l_y=SkeletonFrame.SkeletonData[1].SkeletonPositions[NUI_SKELETON_POSITION_HAND_LEFT].y;
        float hand_l_z=SkeletonFrame.SkeletonData[1].SkeletonPositions[NUI_SKELETON_POSITION_HAND_LEFT].z;   
               
       
            //right hand
        float hand_r_x=SkeletonFrame.SkeletonData[1].SkeletonPositions[NUI_SKELETON_POSITION_HAND_RIGHT].x;
        float hand_r_y=SkeletonFrame.SkeletonData[1].SkeletonPositions[NUI_SKELETON_POSITION_HAND_RIGHT].y;
        float hand_r_z=SkeletonFrame.SkeletonData[1].SkeletonPositions[NUI_SKELETON_POSITION_HAND_RIGHT].z;
        //if (hand_l_x!=0 && hand_l_y!=0)
        if (hand_l_x!=0 && hand_r_z!=0 )
        {
            //printf("HandLeft X=%4.2f    ",hand_l_x,"HandLeft Y=%4.2f    ",hand_l_y,"HandLeft Z=%4.2f    \r\n",hand_l_z);
            //printf("HandRight X=%4.2f    ",hand_r_x,"HandRight Y=%4.2f    ",hand_r_y,"HandRight Z=%4.2f    \r\n",hand_r_z);   
            printf("HandLeft X=%4.2f    ",hand_l_x,"HandLeft Y=%4.2f    ",hand_l_y,"HandLeft Z=%4.2f    \r",hand_l_z);
            printf("HandRight X=%4.2f    ",hand_r_x,"HandRight Y=%4.2f    ",hand_r_y,"HandRight Z=%4.2f    \r",hand_r_z);   

        }
           
            }        
        NuiTransformSmooth(&SkeletonFrame,NULL);
       
        }
      
    system ("pause");                       
    return 0;
    }

    Sorry for stupid questions. But hope someone will help me.       

    Friday, July 1, 2011 10:32 AM
  • Your program won't work by assuming that the recognized skeleton is on the skeleton data structure at index 1. You do indeed need to iterate through all elements of SkeletonFrame.SkeletonData array, from 0 to NUI_SKELETON_COUNT and look for skeletons with eTrackingState  == NUI_SKELETON_TRACKED. The skeleton recognition algorithm doesn't guarantee that the first recognized skeleton will be in slot 0 or 1 or whichever. It could end up anywhere. The only level of consistency that is guaranteed is that, once a skeleton is recognized, it will keep the same slot and tracking ID until the tracking algorithm loses the skeleton when, e.g.: user moves away from screen or too close to screen (skeleton recognition only works for full human body detected within camera frame).

    I know this makes things harder for you, but the skeleton tracking is done at a lower level than the higher level concept of an application "player", and the association between a skeleton slot and/or a tracking ID with a conceptual player needs to be done by the application. Does that make sense?

    Eddy


    I'm here to help
    Friday, July 1, 2011 11:39 PM
  • Eddy,  i finally solved the problem. The key was the NUI_SKELETON_COUNT function. Now the program works fine with fast player detection. :)

    For any interested i ll put the code, without any warranties. Parts of it came from post of Bogdans( thank!)

    Hope someone can use it.

     

    #include <windows.h>
    #include  "MSR_NuiApi.h"
    #include <string>
    #include "MSRKinectAudio.h"
    #include "MSR_NuiImageCamera.h"
    #include "MSR_NuiProps.h"
    #include "MSR_NuiSkeleton.h"
    #include "MSRKinectAudio.h"
    #include "NuiImageBuffer.h"
    #include "resource.h"
    #include <mmsystem.h>
    #include "SkeletalViewer.h"
    int x;
    int null = 0;
    int m_eins = 1;
    HRESULT hr = S_OK;
    int nCount = 0;
       

    HANDLE        m_hNextDepthFrameEvent;
        HANDLE        m_hNextVideoFrameEvent;
        HANDLE        m_hNextSkeletonEvent;
        HANDLE        m_pDepthStreamHandle;
        HANDLE        m_pVideoStreamHandle;



    int connect()
    {
        hr = NuiInitialize( NUI_INITIALIZE_FLAG_USES_SKELETON);
      if( FAILED( hr ) )
      {
        printf ("nicht initialisiert!! KINECT verbunden? \n");
        system("pause");
      
      }
      else
      {
        // m_hNextSkeletonEvent = NULL;
      //m_hNextSkeletonEvent = CreateEvent( NULL, TRUE, FALSE, NULL );
     
        NUI_SKELETON_FRAME SkeletonFrame;
      NuiSkeletonTrackingEnable;

      //NuiSkeletonTrackingEnable( m_hNextSkeletonEvent, 0 );
      }
         
     
    return 0;
    }



    int main()
    {
        connect();
       
        NUI_SKELETON_FRAME SkeletonFrame;
    //hr = NuiSkeletonGetNextFrame( 0, &SkeletonFrame );
      while (true)
        {
            NuiSkeletonGetNextFrame( 0, &SkeletonFrame );
           

        bool bFoundSkeleton = true;
        for( int i = 0 ; i < NUI_SKELETON_COUNT ; i++ )//////////////////hier werden mehrere Skeletons abgefragt!!!!!!!!!!!!!!!
        {
           
            ////////////////////////hier wird nur ein skeleton abgefragt!!!!!!!
            if( SkeletonFrame.SkeletonData[i].eTrackingState == NUI_SKELETON_TRACKED )////////////////prüft ob skeletondaten verfügbar sind
            {
                //bFoundSkeleton = false;
               
            //printf(" Found skeleton");

       
        float hand_l_x=SkeletonFrame.SkeletonData[i].SkeletonPositions[NUI_SKELETON_POSITION_HAND_LEFT].x;
        float hand_l_y=SkeletonFrame.SkeletonData[i].SkeletonPositions[NUI_SKELETON_POSITION_HAND_LEFT].y;
        float hand_l_z=SkeletonFrame.SkeletonData[i].SkeletonPositions[NUI_SKELETON_POSITION_HAND_LEFT].z;   
               
       
            //right hand
        float hand_r_x=SkeletonFrame.SkeletonData[i].SkeletonPositions[NUI_SKELETON_POSITION_HAND_RIGHT].x;
        float hand_r_y=SkeletonFrame.SkeletonData[i].SkeletonPositions[NUI_SKELETON_POSITION_HAND_RIGHT].y;
        float hand_r_z=SkeletonFrame.SkeletonData[i].SkeletonPositions[NUI_SKELETON_POSITION_HAND_RIGHT].z;
        //if (hand_l_x!=0 && hand_l_y!=0)
        if (hand_l_x!=0 && hand_r_z!=0 )
        {
            //printf("HandLeft X=%4.2f    ",hand_l_x,"HandLeft Y=%4.2f    ",hand_l_y,"HandLeft Z=%4.2f    \r\n",hand_l_z);
            //printf("HandRight X=%4.2f    ",hand_r_x,"HandRight Y=%4.2f    ",hand_r_y,"HandRight Z=%4.2f    \r\n",hand_r_z);   
            printf("HandLeft X=%4.2f    ",hand_l_x,"HandLeft Y=%4.2f    ",hand_l_y,"HandLeft Z=%4.2f    \r",hand_l_z);
            printf("HandRight X=%4.2f    ",hand_r_x,"HandRight Y=%4.2f    ",hand_r_y,"HandRight Z=%4.2f    \r",hand_r_z);   

        }

        //////////////////////////////////////////////////////////////       
        else
                    {
                        
                NuiSkeletonGetNextFrame( 0, &SkeletonFrame );
                    }
           
            }       
        }

        // no skeletons!
        //
        if( bFoundSkeleton )
        {
           //    printf("NOT FOUND");
            //system("pause");
            //return 0;
        }

        // smooth out the skeleton data
        NuiTransformSmooth(&SkeletonFrame,NULL);
        // we found a skeleton, re-start the timer
       //NuiSkeletonTrackingEnable;

        }
     

                           

                         

    system ("pause");                       
    return 0;
    }

     

     

    • Marked as answer by ivan_balvan Monday, July 4, 2011 12:06 PM
    Monday, July 4, 2011 12:06 PM
  • Im working on something that might involve that code. is their a way to write this in c#? Its a little easier to digest then c++ although i understand the above code someone else might not.
    Tuesday, July 5, 2011 4:20 PM
  • Take a look at the SkeletalViewer sample that uses C#. The nui_SkeletonFrameReady() function uses the Joints array to draw the skeleton.
    Tuesday, July 5, 2011 8:27 PM
  • Hello,

    Sorry noob question

    • Edited by Sambafr Friday, July 8, 2011 4:33 PM answered
    Friday, July 8, 2011 1:22 PM
  • Hi Ivan,

    thanks for your code. But one question, I just tried to run it and I have some troubles with the linker, which files yo included there?

    You would really help me

    Thanks

    Thursday, July 28, 2011 3:14 AM
  • or do you have the project details? like additional lib, dlls, and that stuff?
    Thursday, July 28, 2011 4:58 AM
  • I felt like the above code was useful, but it had a lot of guff in it making it near unreadable. Also the coding standards were rather lucid and disturbing. I took the liberty of rewriting it to a readable degree, I hope this helps:

    #include "MSR_NuiApi.h"

    #include "Mrs_NuiImageCamera.h"

    #include "MRS_NuiProps.h"

    #include "MRS_NuiSkelton.h"

    #include "MRSKinectAudio.h"

     

    /* all the includes that come with the kinect c++ api */

     

    int isconected = 0;

     

    HRESULT hr = S_OK;

     

    HANDLE m_hNextDepthFrameEvent;

    HANDLE m_hNextVideoFrameEvent;

    HANDLE m_hNextSkeltonEvent;

    HANDLE m_pDepthStreamHandle;

    HANDLE m_pVideoStreamHandle;

     

    /* Main loop that searches for a kinect then a skeleton. */

     

    int main(){

     

    connect();

    while(isconnected == 1 && NUI_SKELETON_COUNT == 1){

    if(SkeletonFrame.SkeletonData[1].eTrackingState == NUI_SKELETON_Tracked){

     

    printf("Skeleton found. \n");

    righthand();

    lefthand();

    if(righthandY != 0 | lefthandY != 0){

     

    printall();

    }else{

    NuiSkeletonGetNextFrame(0, &SkeletonFrame);

    }

    }else{

     

    NuiTransformSmooth(&SkeletonFrame, null);

    printf("No Skeleton found. \n");

    connect();

    }

    }

    }

     

    /* Checks if the kinect is 'kinected' */

     

    void connect(){

     

    hr = NuiInitialize(NUI_INITIALIZE_FLAG_USES_SKELETON);

    if(FAILED(hr)){

     

    printf("No Kinect found. \n");

    isconnected = 0;

    return;

    }else{

     

    NUI_SKELETON_FRAME SkeletonFrame;

    NuiSkeletonTrackingEnable;

    }

     

    printf("Kinect found. \n");

    isconnected = 1;

    return;

    }

     

    /* Retrieves the X, Y, and Z values of your right hand. */

     

    void righthand(){

    float righthandX = SKeletonFrame.SkeletonData[1].SkeletonPositions[NUI_SKELETON_POSITION_HAND_RIGHT].x;

    float righthandY = SKeletonFrame.SkeletonData[1].SkeletonPositions[NUI_SKELETON_POSITION_HAND_RIGHT].y;

    float righthandZ = SKeletonFrame.SkeletonData[1].SkeletonPositions[NUI_SKELETON_POSITION_HAND_RIGHT].z;

    }

     

    /* Retrieves the X, Y, and Z values of your left hand. */

     

    void lefthand(){

    float lefthandX = SKeletonFrame.SkeletonData[1].SkeletonPositions[NUI_SKELETON_POSITION_HAND_LEFT].x;

    float lefthandY = SKeletonFrame.SkeletonData[1].SkeletonPositions[NUI_SKELETON_POSITION_HAND_LEFT].y;

    float lefthandZ = SKeletonFrame.SkeletonData[1].SkeletonPositions[NUI_SKELETON_POSITION_HAND_LEFT].z;

    }

     

    /* Prints right and left hand x, y, and z cordiantes to the terminal. */

     

    printall(){

     

    printf("righthandX = %d, righthandY = %d, righthandZ = %d. \n", righthandX, righthandY, righthandZ);

    printf("lefthandX = %d, lefthandY = %d, lefthandY = %d. \n", lefthandX, lefthandY, lefthandZ);

    }

    Friday, December 16, 2011 5:37 AM
  • I don't get where do you whrite this code. Please hepl me!!!
    Thursday, June 28, 2012 12:55 PM
  • I understand 90% of the code posted here (I'm using the first version from ivan_balvan). Can someone please explain what is going on in this line?

    printf("HandLeft X=%4.2f    ",hand_l_x,"HandLeft Y=%4.2f    ",hand_l_y,"HandLeft Z=%4.2f    \r\n",hand_l_z);

    Specifically, what is the significance of the "%4.2f?"

    Forgive me for the rookie question...
    Tuesday, September 25, 2012 4:30 PM
  • I understand 90% of the code posted here (I'm using the first version from ivan_balvan). Can someone please explain what is going on in this line?

    printf("HandLeft X=%4.2f    ",hand_l_x,"HandLeft Y=%4.2f    ",hand_l_y,"HandLeft Z=%4.2f    \r\n",hand_l_z);

    Specifically, what is the significance of the "%4.2f?"

    Forgive me for the rookie question...

    Looks like a placeholder for a value to me.

    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - "Sherlock holmes" "speak softly and carry a big stick" - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda. Blog - http://www.computerprofessions.co.nr

    Tuesday, September 25, 2012 7:27 PM
  • Not sure exactly how it works but I know that the %4.2 determines how many decimal places the program displays for each coordinate.

    I tried a couple things and got the coordinates into the correct format for my program. I'm not sure exactly what I did but it works! I have no need to experiment any further but I'll share what I did for anyone who's interested:

    here is the original line from ivan_balvan: (only change I made is it now tracks the head instead of the left hand)

    float    head_x= SkeletonFrame.SkeletonData[i].SkeletonPositions[NUI_SKELETON_POSITION_HEAD].x;

    //then later after some more code:

    printf("Head X=%4.2f \n\n",head_x,"Head Y=%4.2f \n\n",head_y,"Head Z=%4.2f \n\n",head_z);

    Here's what I did:

    float    head_x= (((SkeletonFrame.SkeletonData[i].SkeletonPositions[NUI_SKELETON_POSITION_HEAD].x)*100)+500);

    printf("Head X=%4.0f \n\n",head_x,"Head Y=%4.2f \n\n",head_y,"Head Z=%4.2f \n\n",head_z);

    What this does is it changes the format of the coordinate from a decimal to a 3 digit integer. The values are always 3 digits (becuse I multiplied by 100) and always positive (because I added 500).

    If anyone can provide a more thorough explanation of the "%4.2" in these lines I'd be interested to learn what I actually did ;)

    Friday, September 28, 2012 5:49 PM