none
How to write the simplest code to get x,y,z and r,g,b of each pixles using C++

    Question

  • Isaw there are little examples using c++ instead of c#. I just want to run a main function inside a console project in vs c++, and read out the position and color information of each pixels from Kinect. How could I do it? It's hard for me  to understand the c++ skeletonViewer example, because it seems so complicated to me...

    If anyone can give me some really concrete coding idea, I would really appreciate it.

    Saturday, July 09, 2011 1:50 AM

Answers

  •  

    Hi Lizziey, 

    You say you want to read out the position and color information of each pixel - how exactly do you want that read out?  Do you plan to display it visually like they do in the skeletonViewer samples?  Do you want be able to capture 'snapshots' and store the information into image or text files?   

     

    Most of the complication in the samples is setting up the visualization, the 'event grabbers' and the user interface. 

    • Initialize the kinect using NuiInitialize 
    • register an event (and store the event handle) using CreateEvent - How many events you want is up to you - you'll probably want an event for the depth image stream and for the RGB camera stream and for the Skeleton information. 
    • Call for skeleton tracking enable NuiSkeletonTrackingEnable if you plan to use skeleton tracking stuff. Use your handle for the skeleton CreateEvent
    • Use that handle created from CreateEvent to open a stream (and store a handle to this stream) using NuiImageStreamOpen (you need to call this twice - once for depth or depth+skeleton tracking information and once for RGB camera information

    Now what needs to happen is your system has to constantly poll the events you registered, to see if they have occurred.   Generally this is done in a loop that goes forever until you tell it to stop by some external command.  This could be done in its own thread so that you can build up handy UI stuff, but it doesn't have to be if you're looking for simple. 

     

    • Set up an array of your event handles - these are the handles you created with CreateEvent. 
    • Force the loop
    • inside the loop call to WaitForMultipleObjects using your event handle array 
    • Check the result of WaitForMultipleObjects with a switch statement or series of if/else or however.  
    • WaitForMultipleObjects will let you know if any of your events have completed. For instance, if your Depth Image Stream Event handle has completed then the index to that handle in your event handle array will be returned, that means the kinect has completed processing a 'snapshot' of the depth information and you can now access and make use of it. 
    • Process the data as appropriate - you can mostly pull this part straight out of the skeletonviewer sample functions. 

     

    When you're done 

    • close the handles of your events using CloseHandle
    • call NuiShutdown  to finish shutting down the kinect

     

    I think I covered the essentials there - giving a straight code example is difficult because there is no indication of what kind of user interface you plan or what you want to do with the data once you have access to it!  Best of luck :)

     


    Susan
    Monday, July 11, 2011 5:32 PM

All replies

  • Under documentaion on the Kinet SDK main page is a programming guide and a walkthrough of the SkeletonViewer app in both C++ and C#. I would suggest read those, then ask specific questions for clearification. If you don't understand something plenty of people would be more than happy to explain it to you. They only get annoyed if the answer to the question is clearly given in the documentation so it's obvious you didn't read it.

    Saturday, July 09, 2011 6:52 AM
  •  

    Hi Lizziey, 

    You say you want to read out the position and color information of each pixel - how exactly do you want that read out?  Do you plan to display it visually like they do in the skeletonViewer samples?  Do you want be able to capture 'snapshots' and store the information into image or text files?   

     

    Most of the complication in the samples is setting up the visualization, the 'event grabbers' and the user interface. 

    • Initialize the kinect using NuiInitialize 
    • register an event (and store the event handle) using CreateEvent - How many events you want is up to you - you'll probably want an event for the depth image stream and for the RGB camera stream and for the Skeleton information. 
    • Call for skeleton tracking enable NuiSkeletonTrackingEnable if you plan to use skeleton tracking stuff. Use your handle for the skeleton CreateEvent
    • Use that handle created from CreateEvent to open a stream (and store a handle to this stream) using NuiImageStreamOpen (you need to call this twice - once for depth or depth+skeleton tracking information and once for RGB camera information

    Now what needs to happen is your system has to constantly poll the events you registered, to see if they have occurred.   Generally this is done in a loop that goes forever until you tell it to stop by some external command.  This could be done in its own thread so that you can build up handy UI stuff, but it doesn't have to be if you're looking for simple. 

     

    • Set up an array of your event handles - these are the handles you created with CreateEvent. 
    • Force the loop
    • inside the loop call to WaitForMultipleObjects using your event handle array 
    • Check the result of WaitForMultipleObjects with a switch statement or series of if/else or however.  
    • WaitForMultipleObjects will let you know if any of your events have completed. For instance, if your Depth Image Stream Event handle has completed then the index to that handle in your event handle array will be returned, that means the kinect has completed processing a 'snapshot' of the depth information and you can now access and make use of it. 
    • Process the data as appropriate - you can mostly pull this part straight out of the skeletonviewer sample functions. 

     

    When you're done 

    • close the handles of your events using CloseHandle
    • call NuiShutdown  to finish shutting down the kinect

     

    I think I covered the essentials there - giving a straight code example is difficult because there is no indication of what kind of user interface you plan or what you want to do with the data once you have access to it!  Best of luck :)

     


    Susan
    Monday, July 11, 2011 5:32 PM
  • Hi, 

    im also new on the field of Kinectprogramming and trying to access skeletal data. When i found that forum i ve been kind of happy and tried to figure out what susan described. I guess i covered all essentials, but when im in the force while loop... WaitForManyObjects returns just -1.

    Any Ideas what could be wrong? The Code is kind of straight forward copied/ paste from skeletalViewer app. Maybe i missed something? 

    Greetings

    #include <windows.h>
    #include "MSR_NuiApi.h"
    #include <stdio.h>
    
    //Events ...
    HANDLE m_hNextDepthFrameEvent;
    HANDLE m_hNextVideoFrameEvent;
    HANDLE m_hNextSkeletonEvent;
    
    //EventHandle
    HANDLE m_pVideoStreamHandle;
    HANDLE m_pDepthStreamHandle;
    
    // thread handling
    HANDLE m_hThNuiProcess;
    HANDLE m_hEvNuiProcessStop;
    
    
    int main(int argc, char* argv){
    
    	HRESULT        hr;
      m_hNextDepthFrameEvent = CreateEvent( NULL, TRUE, FALSE, NULL );
      m_hNextVideoFrameEvent = CreateEvent( NULL, TRUE, FALSE, NULL );
      m_hNextSkeletonEvent = CreateEvent( NULL, TRUE, FALSE, NULL );
    
      hr = NuiInitialize( 
        NUI_INITIALIZE_FLAG_USES_DEPTH_AND_PLAYER_INDEX | NUI_INITIALIZE_FLAG_USES_SKELETON | NUI_INITIALIZE_FLAG_USES_COLOR);
      if( FAILED( hr ) ) printf("Initialize verkackt \ n");
      
    	hr = NuiSkeletonTrackingEnable( m_hNextSkeletonEvent, 0 );
      if( FAILED( hr ) ) printf("TrackingEnable verkackt \n");
      
    	 hr = NuiImageStreamOpen(NUI_IMAGE_TYPE_COLOR, NUI_IMAGE_RESOLUTION_640x480,
    							 0, 2, m_hNextVideoFrameEvent, &m_pVideoStreamHandle );
      if( FAILED( hr ) ) printf("VideoStreamHandle verkackt \n");
      
    
      hr = NuiImageStreamOpen(NUI_IMAGE_TYPE_DEPTH_AND_PLAYER_INDEX, NUI_IMAGE_RESOLUTION_320x240,
    							0, 2, m_hNextDepthFrameEvent, &m_pDepthStreamHandle );
      if( FAILED( hr ) ) printf("DepthStreamHanle verkackt \n");
      
      HANDLE	hEvents[4];
      int		nEventIdx;
    
    	// Configure events to be listened on
      hEvents[0] = m_hEvNuiProcessStop;
      hEvents[1] = m_hNextDepthFrameEvent;
      hEvents[2] = m_hNextVideoFrameEvent;
      hEvents[3] = m_hNextSkeletonEvent;
    	
    	// Main thread loop
      while(1)
      {
        // Wait for an event to be signalled
        nEventIdx=WaitForMultipleObjects(sizeof(hEvents)/sizeof(hEvents[0]),hEvents,FALSE,100);
        // If the stop event, stop looping and exit
        if(nEventIdx==0)
          break;
    		// Process signal events
        switch(nEventIdx)
        {
          case 1:
            printf("Do Stuff with DepthFrame");
            break;
    
          case 2:
             printf("Do Stuff with RGB");
            break;
    
          case 3:
             printf("Do Stuff with Skelett");
            break;
        }
      }
      return hr;
    }
    

    Saturday, July 23, 2011 8:16 PM
  • Hi Lizziey,

    I looked at the sample too. But it is overloaded.

    I found a realy nice post here in the forum. It's realy simple (init and loop). You only have to import the MSRKinectNUI.lib.

    Saturday, July 23, 2011 8:41 PM
  • Hehe just wanted to post the same link =) 

    also just got it working.

    Thx for fast reply.

    Greetings

    Saturday, July 23, 2011 8:45 PM
  • Maybe try calling GetLastError and FormatMessage to get an explaination for why it failed.
    Saturday, July 23, 2011 8:52 PM
  • // Wait for an event to be signalled
    		nEventIdx = WaitForMultipleObjects(4,hEvents,FALSE,100);
    		//nEventIdx = WaitForSingleObject(m_hNextSkeletonEvent,1000);
    		DWORD err = ::GetLastError();
    		LPVOID lpMsgBuf;
    
    		::FormatMessage( 
    			FORMAT_MESSAGE_ALLOCATE_BUFFER | 
    			FORMAT_MESSAGE_FROM_SYSTEM | 
    			FORMAT_MESSAGE_IGNORE_INSERTS,
    			NULL,
    			GetLastError(),
    			MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT), // Default language
    			(LPTSTR) &lpMsgBuf,
    			0,
    			NULL);
    
    			::MessageBox( NULL, (LPCTSTR)lpMsgBuf, "Error", MB_OK | MB_ICONINFORMATION );
    
    // Free the buffer.
    LocalFree( lpMsgBuf );
    


    When i try to figure out error with GetLastError im getting invalide handle message... im a complete noob in the field of eventprogramming, so that this error statement isnt helping the most. BTW: if i just run with WaitForSingleObject it returns without error message.

    Any ideas?

     

    Greeting 

    Sunday, July 24, 2011 7:22 AM
  • I don't see where you assign a value to this: m_hEvNuiProcessStop. That's actually an event you create to signal the thread processing the kinect sensor data to terminate. With a GUI you would need to run the wait for kinect data in a seperate thread so the interface continues responding to user input. With a console app it isn't as important though you likely should have a way for the user to signal to terminate the app. So, for now, eliminate m_hEveNuiProcessStop and just wait on the other three events.

    Likely you'll want to spawn a thread like the example that has the wait in it while the main thread display a message like "Press any key to exit" and waits for user input. Maybe give it a simple command line interface to like capture frames to a bitmap, i.e. "capture depth to my.bmp". Maybe a "display joint positions" and such so you actually get some feedback from the app.

    Sunday, July 24, 2011 3:41 PM
  • Can anyone post how to link the libraries or include them in the project? I have been getting all kinds of error with VS not able to open the include files. I cannot find an answer on Google either. 

    BTW, if you can open your entire project file. That will be great.

    Thanks.

    Monday, April 09, 2012 5:33 AM
  • Can anyone post how to link the libraries or include them in the project? I have been getting all kinds of error with VS not able to open the include files. I cannot find an answer on Google either. 

    BTW, if you can open your entire project file. That will be great.

    Thanks.

    Hey,

    this is an old post and there have been some changes.

    Here you can find how to use Visual Studio with Kinect.

    Monday, April 09, 2012 9:45 AM
  • Hi Susan Simon

        Sorry to trouble you,I think i meet the same problem as yours.Had you solved it yet? I need your help.

        I want to show point clouds(X,Y,Z) with color(R,G,B) in OpenGL,the data gotten through kinect in SDK 1.6.But I don't know how to change depth to XYZ ,and how to align color information RGB with depth information XYZ( I also use the function NuiImageGetColorPixelCoordinatesFromDepthPixel,but I don't know how to implement it ).This question confused me few days,hope you will help me.Thanks a lot.I am looking forward to you reply.Thanks again!

    Thanks

    Tuesday, November 27, 2012 2:44 AM