none
KinectRegion pointer click by hold RRS feed

  • Question

  • The current KinectRegion click event is generated by moving your hand forward over a control, mimicking a press. In some older applications, the way to do a click was to hold the pointer over a control until a progress circle was completed.

    We are porting an old Kinect1 application to Kinect2, and, while the new pointer "press" event is working fine, our current user base is already used to hold instead of press, and we're having a lot of trouble trying to train them.

    Is it possible to tell the KinectRegion pointer to generate click events when holding over a control instead of pressing?

    Thanks in advance


    Vicente Penades


    Monday, March 16, 2015 3:26 PM

Answers

  • I have the same issue and i found a way around. Is not the most beautiful solution but it worked for me. I noticed that the buttons changed when the handpointer hover on a button, but the OnMouseEnter event does not trigger. So i figured.. What if i fake the onHover event through the Mouse Pointer.

    We executed the code on window load.

    // if loaded executed
    Loaded += KinectPointerPointSample_Loaded;


    Then we listen to the handpointer and place the mouse on the position of the handpointer

    void KinectPointerPointSample_Loaded(object sender, RoutedEventArgs e)
            {
                // Listen to Kinect pointer events
                KinectCoreWindow kinectCoreWindow = KinectCoreWindow.GetForCurrentThread();
                kinectCoreWindow.PointerMoved += kinectCoreWindow_PointerMoved;
            }
    
            private void kinectCoreWindow_PointerMoved(object sender, KinectPointerEventArgs args)
            {
                KinectPointerPoint kinectPointerPoint = args.CurrentPoint;
                
                bool isEngaged = kinectPointerPoint.Properties.IsEngaged;
    
                if (isEngaged)
                {
    
                    System.Drawing.Point mousePt = new System.Drawing.Point((int)(kinectPointerPoint.Position.X * kinectRegion.ActualWidth - 80), (int)(kinectPointerPoint.Position.Y * kinectRegion.ActualHeight));
                    System.Windows.Forms.Cursor.Position = mousePt;
                }            
    
            }

    Noticed that the mouse pointer does not have the exact same position of the handpointer, i have tested that with negative results. Thats why i placed the mouse pointer slightly left to the handpointer (80 pixs to be exact). You can play around depending on your project. Finally we hide the mouse pointer with the following code:

    this.Cursor = Cursors.None;

    Now we can use the OnMouseEnter and OnMouseLeave events...

    Hope this helps. Greets

    Wednesday, March 9, 2016 8:26 AM

All replies

  • I don't think you can change this (at least, not simply, as a feature). Interactions for KinectRegions are pretty "closed".

    Besides, I think this interaction was dropped from the interaction model some time ago. I can't remember "click by hold" as a part of the later versions (1.7, 1.8) of Kinect for Windows v1 SDK.

    Monday, March 16, 2015 6:33 PM
  • Hover is not a good interaction model and a lot of user research was put into the new model based on that feedback. Review the Human Interface Guidelines doc for more information on making UI decisions.

    http://download.microsoft.com/download/6/7/6/676611B4-1982-47A4-A42E-4CF84E1095A8/KinectHIG.2.0.pdf


    Carmine Sirignano - MSFT

    Monday, March 16, 2015 7:18 PM
  • This might be right for most users, but... our user base is people with physical and/or cognitive disabilities, and with that kind of users group we only need very basic interactions.

    Some patients might have limited arm movement, so they're able to point to a button, but unable to perform the "press" gesture corectly, a good example of this is a patient with Parkinson.

    Other patients might have cognitive limitations and are unable to remember to "press" after pointing to the right answer.

    In both cases, we found that click by hold is a good enough solution for this specific users group.


    Vicente Penades

    Monday, March 30, 2015 8:02 AM
  • I understand your problem... I suppose Human Interface Guidelines are developed for "average" users and they cannot cover every situation so I don't think Microsoft is going to change this.

    If this is critical for your app, maybe you can try some workaround... I cannot check it right now but buttons are aware of Kinect Hand Pointer going in and out since they change visual state "on hover". Maybe you could use this events and a timer to trigger clicks. As I said I'm not sure if this will work, it is only something I've just thought.

    Tuesday, March 31, 2015 11:13 AM
  • Yes, it's quite critical. Our current workaround is to not use the kinect interaction library and hook directly into the skeleton pointer, but that is forcing us to write our own whole pointer pipeline, and custom, non generic WPF controls.

    Our problem with this approach is that, by using non standard controls, it's harder to maintain, and gives a lot of headaches with skinning, etc, in general it's much less productive.

    Anyway, Microsoft has a history of supporting users with all kinds of impairments, that's why the Windows accesibility features are there for different input devices, I don't see why kinect should be different in that aspect.


    Vicente Penades

    Wednesday, April 8, 2015 7:58 AM
  • It was just a "forecast" for the near future made from Carmine answer and from general Microsoft response to many of the requests made by Kinect developers in the transition from v1 to v2.

    I didn't mean to be pesimistic, I hope they can help you with this issue and technology reaches as many people as possible, particularly if it is for making their lives better.


    • Edited by jmmroldan Wednesday, April 8, 2015 8:32 AM
    Wednesday, April 8, 2015 8:32 AM
  • I have the same issue and i found a way around. Is not the most beautiful solution but it worked for me. I noticed that the buttons changed when the handpointer hover on a button, but the OnMouseEnter event does not trigger. So i figured.. What if i fake the onHover event through the Mouse Pointer.

    We executed the code on window load.

    // if loaded executed
    Loaded += KinectPointerPointSample_Loaded;


    Then we listen to the handpointer and place the mouse on the position of the handpointer

    void KinectPointerPointSample_Loaded(object sender, RoutedEventArgs e)
            {
                // Listen to Kinect pointer events
                KinectCoreWindow kinectCoreWindow = KinectCoreWindow.GetForCurrentThread();
                kinectCoreWindow.PointerMoved += kinectCoreWindow_PointerMoved;
            }
    
            private void kinectCoreWindow_PointerMoved(object sender, KinectPointerEventArgs args)
            {
                KinectPointerPoint kinectPointerPoint = args.CurrentPoint;
                
                bool isEngaged = kinectPointerPoint.Properties.IsEngaged;
    
                if (isEngaged)
                {
    
                    System.Drawing.Point mousePt = new System.Drawing.Point((int)(kinectPointerPoint.Position.X * kinectRegion.ActualWidth - 80), (int)(kinectPointerPoint.Position.Y * kinectRegion.ActualHeight));
                    System.Windows.Forms.Cursor.Position = mousePt;
                }            
    
            }

    Noticed that the mouse pointer does not have the exact same position of the handpointer, i have tested that with negative results. Thats why i placed the mouse pointer slightly left to the handpointer (80 pixs to be exact). You can play around depending on your project. Finally we hide the mouse pointer with the following code:

    this.Cursor = Cursors.None;

    Now we can use the OnMouseEnter and OnMouseLeave events...

    Hope this helps. Greets

    Wednesday, March 9, 2016 8:26 AM
  • Hi Alejandro, thanks for your reply

    Yes, we partially solved the issue pretty much like you did; essentially tracking the hand pointer and manually checking whether it was inside the bounds of any of the buttons on screen.

    As you said, it's not a beautiful solution since you have to hack into every button event, which goes against WPF good practices... but well... better than nothing.

    Thanks anyway, I'll tag it as a solution.


    Vicente Penades

    • Proposed as answer by PankajDeharia Monday, May 8, 2017 9:55 AM
    • Unproposed as answer by PankajDeharia Monday, May 8, 2017 9:55 AM
    • Proposed as answer by PankajDeharia Tuesday, October 3, 2017 7:12 AM
    Wednesday, March 9, 2016 5:56 PM
  • Glad i could help!
    Thursday, March 10, 2016 2:25 PM