none
Skeletal tracking reliability discrepancy

    Question

  • I'm just starting to dig into the SDK and I'm confused by a qualitative discrepancy in a couple of the samples.

    The Shape Game (which uses the C# API) seems to do a reasonable job tracking my skeleton; there are a few quirks here and there, but within my expectations.  The Skeletal Viewer sample (using the C++ API - the one I intend to use), in contrast, seems to be doing a much worse job at tracking.

    Periodically it will display the skeleton, but most of the time the window is simply blank.  Looking at the code, this appears to be what will happen if the skeleton is in the "not tracked" state.

    Does anyone have any idea why there would be a difference between the reliability of the skeletal tracking between those two samples?  Thanks.

    Saturday, February 18, 2012 7:52 AM

Answers

  • OK, in case anyone else is interested, looks like this is a "stream-starvation" issue in the Skeletal Viewer sample.  In the Known Issues documentation, I found the following note:

    "The SkeletalViewer sample (C++/Direct2D+GDI), when CPU is stressed, may fail to draw skeletons. Best practice when being notified of a frame available is to check all pending streams for new frames."

    In other words, the coders assumed your machine would be fast enough to process all the depth and color events and still have time to process the skeleton event before new depth/color events come in.

    I changed the Nui_ProcessThread function to use the following approach and it resolved the issue:

        while ( continueProcessing )
        {
            // Wait for any of the events to be signalled
            nEventIdx = WaitForMultipleObjects( numEvents, hEvents, FALSE, 100 );
    
    		if (nEventIdx == WAIT_TIMEOUT)
    			continue;
    		else if (nEventIdx == WAIT_OBJECT_0)
    		{
    			continueProcessing = false;
    			continue;
    		}
    		else if (nEventIdx > WAIT_OBJECT_0 && nEventIdx <= WAIT_OBJECT_0 + (numEvents - 1))
    		{
    			int firstSignaled = nEventIdx - WAIT_OBJECT_0;
    			for (int i = firstSignaled; i < numEvents; i++)
    			{
    				if (WaitForSingleObject(hEvents[i], 0) == WAIT_OBJECT_0)
    				{
    					switch ( i )
    					{
    						case WAIT_OBJECT_0 + 1:
    							Nui_GotDepthAlert();
    							 ++m_DepthFramesTotal;
    							break;
    
    						case WAIT_OBJECT_0 + 2:
    							Nui_GotColorAlert();
    							break;
    
    						case WAIT_OBJECT_0 + 3:
    							Nui_GotSkeletonAlert( );
    							break;
    					}
    				}
    			}
    		}

    • Marked as answer by Aenimated Wednesday, February 29, 2012 3:05 AM
    Wednesday, February 29, 2012 3:00 AM

All replies

  • Hi,

    Does Kinect Explorer work ok?

    What do you mean by the screen goes blank? Which part of the Skeletal Viewer are you referring to?

    Thx -


    - hope this helps - Mauro.

    Saturday, February 18, 2012 4:06 PM
  • Thanks Mauro for the quick response.

    The Kinect Explorer sample works fine, much like the Shape Game sample.  It's only the Skeletal Viewer sample that seems to exhibit this behavior.  I'm specifically referring to the "Skeletal View" display in the center of the dialog.  Once in a while, it will display a skeleton, but it is very intermittent.  Usually, the display remains black.  Here's the relevant code with some added debug output:

    void CSkeletalViewerApp::Nui_GotSkeletonAlert( )
    {
        NUI_SKELETON_FRAME SkeletonFrame = {0};
    
        bool bFoundSkeleton = false;
    
    	HRESULT err = 0;
        if ( SUCCEEDED(err = m_pNuiSensor->NuiSkeletonGetNextFrame( 0, &SkeletonFrame )) )
        {
            for ( int i = 0 ; i < NUI_SKELETON_COUNT ; i++ )
            {
                if( SkeletonFrame.SkeletonData[i].eTrackingState == NUI_SKELETON_TRACKED ||
                    (SkeletonFrame.SkeletonData[i].eTrackingState == NUI_SKELETON_POSITION_ONLY && m_bAppTracking))
                {
                    bFoundSkeleton = true;
                }
            }
        }
    
        // no skeletons!
        if( !bFoundSkeleton )
        {
    		if ( FAILED(err) )
    		{
    			if (err == E_NUI_FRAME_NO_DATA)
    				OutputDebugString( L"No frame data.\r\n" );
    			else
    				OutputDebugString( L"Unexpected error.\r\n" );
    		}
    		else
    			OutputDebugString( L"No skeletons tracked.\r\n" );
    
            return;
        }
    Unsurprisingly, the debug output I'm seeing is "No skeletons tracked".  So the application is simply not tracking the skeleton reliably.  I would chalk it up to being a bit closer to the Kinect sensor than would be ideal (maybe 6 to 8 feet), but the managed sample applications seems to have no trouble with it.  I'm at a loss to explain this.  If you have any ideas, or can suggest any experiments to try, let me know.  Thanks again.

    Sunday, February 19, 2012 5:33 AM
  • OK, in case anyone else is interested, looks like this is a "stream-starvation" issue in the Skeletal Viewer sample.  In the Known Issues documentation, I found the following note:

    "The SkeletalViewer sample (C++/Direct2D+GDI), when CPU is stressed, may fail to draw skeletons. Best practice when being notified of a frame available is to check all pending streams for new frames."

    In other words, the coders assumed your machine would be fast enough to process all the depth and color events and still have time to process the skeleton event before new depth/color events come in.

    I changed the Nui_ProcessThread function to use the following approach and it resolved the issue:

        while ( continueProcessing )
        {
            // Wait for any of the events to be signalled
            nEventIdx = WaitForMultipleObjects( numEvents, hEvents, FALSE, 100 );
    
    		if (nEventIdx == WAIT_TIMEOUT)
    			continue;
    		else if (nEventIdx == WAIT_OBJECT_0)
    		{
    			continueProcessing = false;
    			continue;
    		}
    		else if (nEventIdx > WAIT_OBJECT_0 && nEventIdx <= WAIT_OBJECT_0 + (numEvents - 1))
    		{
    			int firstSignaled = nEventIdx - WAIT_OBJECT_0;
    			for (int i = firstSignaled; i < numEvents; i++)
    			{
    				if (WaitForSingleObject(hEvents[i], 0) == WAIT_OBJECT_0)
    				{
    					switch ( i )
    					{
    						case WAIT_OBJECT_0 + 1:
    							Nui_GotDepthAlert();
    							 ++m_DepthFramesTotal;
    							break;
    
    						case WAIT_OBJECT_0 + 2:
    							Nui_GotColorAlert();
    							break;
    
    						case WAIT_OBJECT_0 + 3:
    							Nui_GotSkeletonAlert( );
    							break;
    					}
    				}
    			}
    		}

    • Marked as answer by Aenimated Wednesday, February 29, 2012 3:05 AM
    Wednesday, February 29, 2012 3:00 AM