Common NUI Problems and FAQ

Question

This is a listing of answers to Frequently Asked Questions and workarounds for common problems encountered while using the Kinect NUI API.

 

1.1. Problem: NuiInitialize fails with 0x80080014 error code in C++
1.2. Runtime.Initialize throws InvalidOperationException in .NET
Solution: Make sure Kinect device is plugged in to USB port in computer and also connected to power supply.

2.1. Problem: NuiImageStreamOpen fails in C++
 Solution:
A. Make sure desired stream was properly initialized by specifying appropriate flag in NuiInitialize
B. Make sure that specified {image type, image resolution} parameters correspond to one of the supported combinations.

2.2. VideoStream.Open or DepthStream.Open throw InvalidOperationException in .NET
 Solution:
A. Make sure desired stream was properly initialized by specifying appropriate flag in Runtime.Initialize
B. Make sure that specified {image type, image resolution} parameters correspond to one of the supported combinations.

3. Question: Is there an easy way to verify if the Kinect Drivers were correctly installed?
Answer:
I. Plug in kinect
II. Open Device Manager (Control Panel|System and Security|System|Device Manager)
III.  There should be a device group called "Microsoft Kinect" that lists "Microsoft Kinect Audio Array Control", "Microsoft Kinect Camera" and "Microsoft Kinect Device" devices. Also, under "Sound, video and game controllers" there should be a device called "Kinect USB Audio". If all 4 of these devices are listed, then installation was successful.

4. Question: Can I use Kinect device as a camera for conferencing applications?
Answer: No, this is not supported in the current release.

5.1. Problem: My webcam|<other USB device> stopped working after installing Kinect SDK
5.2. Problem: Multiple Kinect devices are not being detected correctly when simultaneously connected
Solution: There is currently a limitation where Kinect devices might not work reliably when they are connected to the same USB host controller as other USB devices, such as webcams or other Kinect devices. Typically PCs come with one USB controller in the front and one in the back, so this kind of problem is often resolved when Kinect is connected to the front of the PC and other devices to the back of PC or vice versa.

6. Problem: Exception when running application because INuiInstanceHelper.dll not found
Characteristics: Typically happens after compiling and running first managed application after installing SDK
Solution: Development environment may not have been properly updated to recognize installed binaries if Visual Studio was open during installation. First try closing and reopening visual studio and if that doesn't work, try rebooting machine.

7. Question: Can SDK APIs be used to track:
  7.1. Upper body?
  7.2. Fingers?
  7.3. Other objects?
The NUI API in Kinect for Windows SDK Beta only provides direct support for tracking joint skeletons representing a whole human body in a visible frame, with joints such as hips, knees, elbows, spine and kneck. There is some tolerance in that usually a skeleton will be recognized if a human body is visible from knees to neck, with most of arms also visible and, in that case, the skeleton frame will be reported as clipped via SkeletonData.Quality property. However, recognition of upper body, fingers or other objects is not provided out of the box.
That being said, it should be possible to implement other kinds of object tracking by using information available from the NUI depth stream, and we know there is interest around this area so we will link to samples if we find good ones provided by community.

8. Question: What is the format for depth stream data?
Each pixel is represented by 2 bytes arranged in little endian order. The detailed layout of the 16-bit data depends on the kind of depth stream being used, but in all cases 12 bits will be used to represent distance from camera (in millimeters) to object perceived at pixel.
  8.1. RuntimeOptions.UseDepth (managed) or NUI_INITIALIZE_FLAG_USES_DEPTH (C++)
  Least significant 12 bits will contain distance from camera (in millimeters). Remaining bits are unused.
  8.2. RuntimeOptions.UseDpethAndPlayerIndex (managed) or NUI_INITIALIZE_FLAG_USES_DEPTH_AND_PLAYER_INDEX (C++)
  Least significant 3 bits will contain index of player detected at pixel , most significant bit is unused and remaining 12 bits contain distance from camera.

9. Question: How can I get skeleton joint positions in world coordinates, in meters?
This is exactly what is provided in SkeletonData.Joints[i].Position property (or NUI_SKELETON_DATA.SkeletonPositions[i] in C++ API), and also what is returned by SkeletonEngine.DepthImageToSkeleton method (NuiTransformDepthImageToSkeletonF in C++). Detailed documentation of skeletal space is in pages 22-23 of programming guide (http://research.microsoft.com/en-us/um/redmond/projects/kinectsdk/docs/ProgrammingGuide_KinectSDK.pdf).
It seems a common misconception that position joint X values will be limited to -1 to +1 range, but it is in fact possible for values smaller than -2 and greater than +2 to be returned for either X or Y coordinates in skeleton space, if user being tracked is far enough away from camera and far enough to left or right from center.

10. Question: How do I get world coordinates corresponding to a pixel in depth stream frame?
Use NuiTransformDepthImageToSkeletonF (in C++) or SkeletonEngine.DepthImageToSkeleton (in C#) methods.
Note that this method expects normalized depth pixel coordinates in 0.0-1.0 range. To obtain normalized coordinate for a pixel you need to divide pixel index in image by image dimensions. E.g.: normalized coordinates for pixel (45,100) in a 320x240 depth image are (45.0/320,100.0/240), which is: (0.140625,0.416667).
Also, the depth value that these functions take is pixel depth, in millimeters, left shifted by 3. So, if variable named depthValue contains results of calculation described in question 8 in this FAQ, then depth passed to DepthImageToSkeleton method should be (depthValue << 3). Not performing this left shift will result in computation errors that are visibly noticeable as parallax effects.

11. Question: How do I get color image coordinates corresponding to a pixel in depth stream frame?
Use NuiImageGetColorPixelCoordinagesFromDepthPixel (in C++) or Camera.GetColorPixelCoordinatesFromDepthPixel (in C#).
Note that the depth value that these functions take is pixel depth, in millimeters, left shifted by 3. So, if variable named depthValue contains results of calculation described in question 8 in this FAQ, then depth passed to GetColorPixelCoordinatesFromDepthPixel method should be (depthValue << 3). Not performing this left shift will result in computation errors that are visibly noticeable as parallax effects.

12. How do I get the joint (x,y,z) coordinates for a tracked skeleton?
  12.1. In C++
  Look at SkeletalViewer C++ sample, installed to C:\Users\Public\Documents\Microsoft Research KinectSDK Samples\NUI\SkeletalViewer\CPP. Specifically, look at how method CSkeletalViewerApp::Nui_DrawSkeleton in file NuiImpl.cpp manipulates the 'NUI_SKELETON_DATA *pSkel' variable and its 'SkeletonPositions' array field in order to draw skeleton segments on screen.
  12.2. In C#
  Look at SkeletalViewer C# sample, installed to C:\Users\Public\Documents\Microsoft Research KinectSDK Samples\NUI\SkeletalViewer\CS. Specifically, look at how nui_SkeletonFrameReady in file MainWindow.xaml.cs manipulates the 'SkeletonData data' variable and its 'Joints' property in order to draw skeleton segments on screen.

13. Where can I find a basic tutorial for using Kinect NUI functionality?
The Kinect For Windows SDK Quickstarts in Channel 9 (http://channel9.msdn.com/series/KinectSDKQuickstarts/) are a great place to start. Documentation, in particular the programming guide can be found here: http://research.microsoft.com/en-us/um/redmond/projects/kinectsdk/guides.aspx
Also, once you install the SDK, you will find a SkeletalViewer sample installed to C:\Users\Public\Documents\Microsoft Research KinectSDK Samples\NUI\SkeletalViewer\CS.