List of issues RRS feed

  • Question

  • This is a list of long standing issues I've been collecting over time, some are just suggestions or requests, some other are plain bugs. For some we've found work arounds, for others we haven't and keep being show stoppers. I just hope some of them are taken into consideration.

    1-  Windows enters in suspend mode after a while

    When we develop a Kinect application, you expect the users to interact with the system using gesture recognition, etc. During that time, the mouse isn't moved and no keys are pressed, and depending on the power saving configuration, the system might think nobody is attending the machine and suspends the window.

    Reproduction: just try some demos around that didn't take this issue in account, the display will suspend after a while.

    Expected behaviour: when the Kinect is running, and tracking skeletons, it should not suspend the display.

    Workaround: manually notify the system we don't want to suspend the system

    2- Reentrancy Exception in Kinect Interaction

    From time to time we get this exception, it's been asked before here. It's quite difficult ro reproduce, I beleave it happens whith two-hand tracking, since we've fell back to just one user-hand tracking we didn't experienced it, I would not consider it a workaround yet.

    3- The kinect core interaction API only handles hand pointer positioning, not hand pressing.

    Hand pointers are part of the core API, but the mechanisms that allow pressing buttons, etc, seems to be exclusive of the WPF interaction controls. I understand handling press gestures requires some knowledge of the context (whether you're over a button area or not) but the same way we have an Engagement API, we could have one for Press.

    Also, we have users with Parkinson and other mobility/ accesibility issues that have difficulties performing the press gesture. For them, the "press by hold" gesture was much easier. But I understand for healthy people an actual press makes more sense.

    4- WPF button press event miss

    In Kinect Interaction WPF applications, sometimes the button fails to trigger the "click" event.

    Reproduction: press a button, and as soon as the the pointer begins doing the "pressed!" animation over the button, move your hand away as fast as possible. The animation will finish, but no action will be done. As you get used to press buttons with the kinect, this issue becomes more common, since you get used to drop your hand faster and faster.

    This is probably happening because the pointer animation blocks for a second, but after that, it expects you to keep your hand pressing the button, and it is not.

    Expected behabiour: when the "pressed!" animation ends, it should trigger the click event regardless of the state of your hand.

    5- WPF Interaction events stop working when the window is child of another window via WIN32

    We had an odd case scenario in which we have a launcher application that spawns processes that display kinect windows. These windows are attached to the launcher window with a User32 .SetParent call , so the Kinect Windows are children of the launcher window. Somehow, this breaks the event handling of the kinect and the whole interaction system is broken.

    6- The API exposes the Accelerometer and Floor height through the skeleton Stream and not as a general value

    Gravity vector and floor height are critical to process the skeleton bodies and at first it makes sense that the gravity vector comes along with the skeleton stream.

    But we found some uses cases in which we need the gravity vector and floor height to process the color-depth streams without requiring the skeletons

    Expected behavior: to get the Gravity and floor height with any stream or upon request.

    7- Coordinate Mapper can only be called from the Kinect calling thread.

    Upon trying to use the coordinate mapper to process some depth images in a multithreaded loop (threading.parallel) , we discovered that the Coordinate Mapper gave us an exception

    Workaround: we created buffer maps for every coordinate transformation we required, but for some operations it's maybe overkill

    Expected behaviour: CoordinateMapper should be called from any thread.

    8- Webcam support

    Yes, this has been requested since the dawn of time and there's several hacks and proofs of concepts around, but nothing official that can be used in a reliable way... and our customers keep asking us if they can use it as a webcam again and again and again.

    Expected behaviour: an in-built webcam driver

    9- Skeleton tracking when the player is facing backwards relative to the Kinect Sensor.

    I'm not asking a full tracking of the player when the user is facing backwards, but just an "engagement" probability hint value that the user is showing his back to the sensor and thus we should not have to pay attention to the tracked skeleton.

    Reproduction: a common behaviour of users is, when they've finished trying a Kinect demo, they leave the playing area by walking sideways or backfacing the sensor; in these cases it makes no sense trying to pay attention to the user, even if its fully visible and fully tracked.

    Expected: We need some sort of "engagement" mechanism to know whether the user is actually engaged.

    That's all, hope you can take these requests in consideration!


    Vicente Penades

    Friday, March 4, 2016 2:17 PM