none
Can you implement Kinect Mouse Concept?

    General discussion

  • hi!

    I got my xbox360 with kinect but it doesn't come with the USB adaptor for the kinect device, so i'll have to wait till i can find one!

    so... because i can't test my code, I'm first writing down the concept for a later implementation, and hoping that maybe someone do it before I do... ehehheh

    this is the concept:

    1. one hand used for the movements up, down, left, right
      1. The mouse will only move if you're hand is at a given distance (you choose what better fits) or closer to the kinect.
      2. When you're hand get far from the kinect the mouse movement will stop and the position will be recorded (ExitPosition) so it's possible to return to that position when you're hand get close again to the kinect.
      3. When you're hand enters the "mouse moving area" again you will store the position (EnterPosition)
        1. Now when your hand moves, at every kinect refresh, you will have the CurrentPosition
        2. Doing EnterPosition-CurrentPosition will get you the TraveledDistance
        3. Now add the TraveledDistance to the ExitPosition and you will get the RelativeMousePosition
        4. Now move the mouse to the RelativeMousePosition that you calculated
    2. the other hand is used for the clicks
      1. Clicking directions:
        1. moving hand UP = left click
        2. moving hand DOWN = RIGHT click
        3. moving hand RIGHT = left and right mouse buttons cliccked togheter
        4. moving hand LEFT = mouse wheel click
      2. How to click:
        1. move the hand to the direction and you will do "down key"
        2. move hand to the starting position and you will do "upper key"
        3. after these 2 events the "mouse click" is done
    3. Pluses:
      1. Make the program work as a tray icon, so it will actually work as a "mouse" and the user could be able to control the "kinect mouse" all over the screen and not just inside the "program window"
    oh... and if you do implement it, share it with us =)

    PS: I'm quite sure that many of the ideas that we are throwing out here in theses forums will be part of the product for the next generation of kinect, and the "mouse function" it's like a basic thing for the ones who don't have or don't want the XBOX console.

    Code Suggestions

    - remember to vote the posts so they can be easly found by others

    User:giasaste

    The following code can be used to control the cursor with your right hand and perform a click when you are holding your left hand up. You can simply change the left hand click and perform a click when your left hand is down, or even perform a mouse-down when your left hand is up and a mouse-up when your left hand is again down

    see the code: version 1

    see the code: improvements

    User:Teemu Tapanila

    I implemented mouse clicking the same way as the selecting is implemented on Kinect Adventure. If you don't move your right hand from a certain location in some amount of time, It will register it as a click.

    see the code

    User:DeV91

    the code is in an external link

    see the code

    User:rcitaliano

    see the code: MouseHook Class







    • Changed type Eddy Escardo-Raffo [MSFT]Microsoft employee Friday, July 01, 2011 2:30 AM this looks like an active discussion with input from various people, rather than a question seeking a specific answer.
    • Edited by rcitaliano Friday, February 10, 2012 9:33 AM specifying the mouse movement
    Tuesday, June 28, 2011 2:49 PM

All replies

  • Hi,

     

    i already started with it in VB.Net. I am interested in using the kinect camera and handle the hand movements in order to control the mouse. 

    First i will try to implement "one hand used for the movements up, down, left, right", and then the "clicking directions". I will post the code here. If you manage to implement something of that it would be cool to see your code posted here.

     

    Ciao

     

    I have problems retrieving the X,Y Coordinates of the hand that should control the cursor. Any ideas?
    Tuesday, June 28, 2011 6:26 PM
  • You guys may want to check out http://kinectmouse.codeplex.com it has a a bunch of the things you are looking for

     

    Rick


    Rick
    Wednesday, June 29, 2011 1:25 AM
  • Hi,

     

    i already started with it in VB.Net. I am interested in using the kinect camera and handle the hand movements in order to control the mouse. 

    First i will try to implement "one hand used for the movements up, down, left, right", and then the "clicking directions". I will post the code here. If you manage to implement something of that it would be cool to see your code posted here.

     

    Ciao

     

    I have problems retrieving the X,Y Coordinates of the hand that should control the cursor. Any ideas?

    Hi =)

     

    really thanks for the interest of implementing it!

    As i stated in the post before, i can't implement anything yet, because i can't connect my kinect to the PC because i bought the xbox360 kinect boundle that came with the xbox slim with the kinect port that already provides the power needed for the kinect, so it didn't came with the "kinect adapter" :( that was sad eeheheh. That's why i can't implement it...because i really don't want to create the code and not be able to test it ehehhehe.

    As soon as i get the adpter ( it's not that easy here in italy ) i'll start to implement it =)!!

    i will be updating the "CONCEPT" as I keep getting ideas, and if any1 has any other ideas i can put it into the post too.

    Ciao =)

    Wednesday, June 29, 2011 7:41 AM
  • This is very cool. I will be looking forward to see where this is going!

    Just got my Kinect hooked up to my PC and will be coding like hell for the next few weeks to see this idea become a reality!

    Wednesday, June 29, 2011 10:14 AM
  • nice to see that we got 1 more into this idea XD

    feel free to change/suggest anything to the concept.

    and post the code, problems, or other concept so we can keep this moving eheheh

    Wednesday, June 29, 2011 10:40 AM
  • Most of this is actually pretty easy, I could implement most of it today. I have already made a project of my own that allows movement of the mouse with the right hand. All of the clicking combinations would be no more than 5 or 6 lines of code added. I can post the code when I get it done.
    Wednesday, June 29, 2011 5:46 PM
  • Hi,

     

    actually Tareq Perez is right, it was actually easy.

    After some experiments i finally managed to implement the features mentioned here. I will code it clean and document it so that anyone can use it.

    I will submit the code (VB)  tomorrow (30.06 Germany)  and also some examples in order to make extensions simple. 

     

    Ciao

    Thursday, June 30, 2011 1:12 AM
  • actually Tareq Perez is right, it was actually easy.

    Wow... so I'm actually curious to see your code =)

    and is it working on all the screen, like a tray icon, or just in a "window"?

    Thursday, June 30, 2011 7:40 AM
  • The cursor program I made works on the whole screen and has a minimized tray icon. I'm curious to see how he did his clicking algorithm as well. It shouldn't be that much code, but still curious regardless.
    Thursday, June 30, 2011 1:44 PM
  • The cursor program I made works on the whole screen and has a minimized tray icon. I'm curious to see how he did his clicking algorithm as well. It shouldn't be that much code, but still curious regardless.
    is your code secret? or will you show it to us? =)
    Thursday, June 30, 2011 1:53 PM
  • The cursor program I made works on the whole screen and has a minimized tray icon. I'm curious to see how he did his clicking algorithm as well. It shouldn't be that much code, but still curious regardless.
    is your code secret? or will you show it to us? =)

    I don't have the mouse clicking algorithm working yet. I know a complicated way to get it done, but I want to see how giasaste did his. I can post the code I have so far if you want.

    Thursday, June 30, 2011 3:36 PM
  • Hi,

    i am aworking on this problem for my bachelor thesis, but i have a differnet approach. I want to use the kinect like a touch interface so that you show on the desired position on the screen and the program knows where you're pointing at.

    This system is already working in it's own environment, so you get only a visual pointing feedback, but the mouse controling should not be so difficult. But there are two problems i am still working on. First one is the "pointing behaviour", i still have to optimize the interpretation of the skeletal data to get a nice and natural feeling when pointing. And the second one is the calibration of the system. The system has to know where the screen is positiont in relation to the camera and i try to make it possible to show thy system the screen through the kinect but that is still very unprecise. For now you have to measure the distances to the corners by hand.

    Overall i can imagine to realease a first version which can control the mouse in the next weeks or months.

     

    Kalle

    Thursday, June 30, 2011 4:37 PM
  • Hi,

    i am aworking on this problem for my bachelor thesis, but i have a differnet approach. I want to use the kinect like a touch interface so that you show on the desired position on the screen and the program knows where you're pointing at.

    This system is already working in it's own environment, so you get only a visual pointing feedback, but the mouse controling should not be so difficult. But there are two problems i am still working on. First one is the "pointing behaviour", i still have to optimize the interpretation of the skeletal data to get a nice and natural feeling when pointing. And the second one is the calibration of the system. The system has to know where the screen is positiont in relation to the camera and i try to make it possible to show thy system the screen through the kinect but that is still very unprecise. For now you have to measure the distances to the corners by hand.

    Overall i can imagine to realease a first version which can control the mouse in the next weeks or months.

     

    Kalle

    How would you even calculate the position of the screen to the camera? It seems like you would have to keep that value constant (like placing the kinect directly in front of the screen).
    Thursday, June 30, 2011 5:15 PM
  • Hi,

     

    Sorry for the delay, i had to work until now. Actually i was surprised that you can simply retrieve the coordinates..of the hands, head and so on. The following code can be used to control the cursor with your right hand and perform a click when you are holding your left hand up. You can simply change the left hand click and perform a click when your left hand is down, or even perform a mouse-down when your left hand is up and a mouse-up  when your left hand is again down. I still have some problems with the screen size. I am not yet sure how to adjust this correctly...still working on it. Nevertheless, here is my first attempt (its my first attempt in VB and Kinect). 

     

    Imports System.IO
    Imports System.Media
    Imports Microsoft.Research.Kinect.Nui
    Imports Microsoft.Research.Kinect.Audio
    Imports Microsoft.Speech.AudioFormat
    Imports Microsoft.Speech.Recognition
    
    
    Public Class Input_Device_Kinect
    
      'These declarations are needed to perform the mouse clicks and cursor movements
      Private Declare Sub mouse_event Lib "user32" (ByVal dwFlags As Integer, ByVal dx As Integer, ByVal dy As Integer, ByVal cButtons As Integer, ByVal dwExtraInfo As Integer)
      Private Const MOUSEEVENTF_LEFTDOWN = &H2
      Private Const MOUSEEVENTF_LEFTUP = &H4
      Private Const MOUSEEVENTF_RIGHTUP = &H10
      Private Const MOUSEEVENTF_ABSOLUTE = &H8000
    
      Private Declare Function SetCursorPos Lib "user32" (ByVal x As Integer, ByVal y As Integer) As Integer
      Private nui As Runtime
      Private Const RecognizerId As String = "SR_MS_en-US_Kinect_10.0"
    
      Public Sub New()
        nui = New Runtime
        nui.Initialize(RuntimeOptions.UseDepthAndPlayerIndex Or RuntimeOptions.UseSkeletalTracking Or RuntimeOptions.UseColor)
    
        nui.VideoStream.Open(ImageStreamType.Video, 2, ImageResolution.Resolution640x480, ImageType.Color)
        nui.DepthStream.Open(ImageStreamType.Depth, 2, ImageResolution.Resolution320x240, ImageType.DepthAndPlayerIndex)
    
        'This is an event handler for the skeleton parts
        AddHandler nui.SkeletonFrameReady, AddressOf nui_SkeletonFrameReady
    
      End Sub
    
      
    
      Private Sub tracking_stop(ByVal sender As Object, ByVal e As EventArgs)
        nui.Uninitialize()
      End Sub
    
      Private Sub nui_SkeletonFrameReady(ByVal sender As Object, ByVal e As SkeletonFrameReadyEventArgs)
    
        Dim skeletonFrame As SkeletonFrame = e.SkeletonFrame
        Dim iSkeleton As Integer = 0
    
        For Each data As SkeletonData In skeletonFrame.Skeletons
          If SkeletonTrackingState.Tracked = data.TrackingState Then
            'Here you call this method with the Joints you want. i.e. JoindID.Head, JoindID.HandRight...
            'From there you can work with these data
            getBodySegment(data.Joints, JointID.HandLeft, JointID.HandRight)
          End If
        Next data
    
      End Sub
    
      Private Function getDisplayPosition(ByVal joint As Joint) As Point
        Return New Point(Screen.PrimaryScreen.Bounds.Width * (joint.Position.X + 1) / 2, Screen.PrimaryScreen.Bounds.Height * (1 - joint.Position.Y) / 2)
      End Function
    
      'Here you simply use the data of the left and right hand for further processing. I call the method setMyCursor() with the data of
      'the right hand to set the position of the cursor and also the method performClick() with the data of the left hand.
      'Here you can also extend the if cases if you send more data(parameters). If ids(i).ToString.Equals("Head") then...for instance.
      Private Function getBodySegment(ByVal joints As Microsoft.Research.Kinect.Nui.JointsCollection, ByVal ParamArray ids() As JointID)
        Dim pointRight As Point
        For i As Integer = 0 To ids.Length - 1
          'Here i have to put the code for the hand
          If ids(i).ToString.Equals("HandRight") Then
            Debug.Print("Right Hand Found")
            pointRight = getDisplayPosition(joints(ids(i)))
            setMyCursor(pointRight.X, pointRight.Y)
          ElseIf ids(i).ToString.Equals("HandLeft") Then
            Debug.Print("Left Hand Found")
            Dim pointLeft As Point = getDisplayPosition(joints(ids(i)))
            'Here we need the coordinates of the right hand in order to know where to 
            'perform the click. The coordinated of the left hand are needed in order 
            'to retrieve if a click should be performed
            performClick(pointRight.X, pointRight.Y, pointLeft.X, pointLeft.Y)
          End If
        Next i
    
      End Function
    
      'Here i simply set the coordinates (in this case Y) to set the click points. In this case i use the hight.
      Private Sub performClick(ByVal pointRX As Integer, ByVal pointRY As Integer, ByVal pointLX As Integer, ByVal pointLY As Integer)
        'Here you can adjust this value (300) to the one that is suitable for your needs
        If pointLY < 300 Then
          'click at position of right hand
          Debug.Print("CLICK #################")
          'Here i simply perform a click.
          mouse_event(MOUSEEVENTF_LEFTDOWN, 0, 0, 0, 0)
          mouse_event(MOUSEEVENTF_LEFTUP, 0, 0, 0, 0)
        End If
      End Sub
    
      Private Sub setMyCursor(ByVal X As Integer, ByVal Y As Integer)
        'Here i set the cursor to the coordinates retrieved from the right hand.
        SetCursorPos(X, Y)
      End Sub
    
    
      
    End Class
    
    <br/>
    
    I will continue and post all improvements. I think that a click concept like in the xbox would be nice. For instance, perform a left click when your right hand is pointing a certain 
    
    area for a period of time and a right click when your left hand is up. Hope i could help a little. 
    

    Thursday, June 30, 2011 10:30 PM
  • giasaste

     

    That's basically what I have right now, I can move the cursor with my right hand and click by pushing my left hand forward. 

    A tip for getting the screen size to work. Check out the coding4fun toolkit: http://channel9.msdn.com/coding4fun/kinect/Coding4Fun-Kinect-Toolkit

     

    It contains a method, joint.ScaleTo(int width,int height) that will scale a joint coordinate to the given resolution.

    That's how I did mine and it works pretty well. You can also pass 2 optional parameters joint.ScaleTo(int width,int height, float skeletonXMax,float skeletonYMax) that will set the bounds on the kinect camera. Basically rather than requiring you to reach to the end of the kinect's vision to get to the right side of the screen, you can limit that right bound to something more practical.

    Friday, July 01, 2011 12:20 AM
  • Hi, thanks for the tip. I tried that before, but for some reason i couldnt achieve better results. I will let you know if i achieve an optimization. Bye
    Friday, July 01, 2011 1:24 AM
  • Hi, Can you give me the above code in C#.?It will be more helpful for me. Regards Subash
    Friday, July 01, 2011 6:07 AM
  • Hi, thanks for the tip. I tried that before, but for some reason i couldnt achieve better results. I will let you know if i achieve an optimization. Bye

    Wow! that was nice.

    I'm still in trouble getting my power supply for the kinect T_T.

    I've added a link to your code to the top post =)

    I was wondering... adding a video to it, would be great eheheheh post it on youtube account =)

    Friday, July 01, 2011 7:42 AM
  • Hi, 

     

    i am glad you liked it.

    Some improvements and extensions.

    1. I utilized the Coding4fun Kinect Toolkit ( http://kinectmouse.codeplex.com ) and used the method ScaleTo() in order to optimize the screen size.

    2. I extended the setMyCursor() method to a timer in order to perform left clicks with your right hand if its in a defined area for a defined period of time.

     

    'You will need to import this. You can find the .dll file in the 
    'following link: http://kinectmouse.codeplex.com .
    'They have done a very nice job there.
    Imports Coding4Fun.Kinect.Wpf
    


    Here is the getBodySegment() method where the optimized screen size is implemented. If you use this, you dont need the getDisplayPosition() method anymore.

    Private Const SkeletonMaxX As Single = 0.6F
    Private Const SkeletonMaxY As Single = 0.4F
    
    'Here you simply use the data of the left and right hand for further processing. I call the method setMyCursor() with the data of
      'the right hand to set the position of the cursor and also the method performClick() with the data of the left hand.
      'Here you can also extend the if cases if you send more data(parameters). If ids(i).ToString.Equals("Head") then...for instance.
      Private Function getBodySegment(ByVal joints As Microsoft.Research.Kinect.Nui.JointsCollection, ByVal ParamArray ids() As JointID)
        Dim pointRight As Point
    
        ' scale those Joints to the primary screen width and height
        For i As Integer = 0 To ids.Length - 1
          'Here i have to put the code for the hand
          If ids(i).ToString.Equals("HandRight") Then
            Dim jointRight As Joint = joints(ids(i))
            'Here i used the Code4fun Kinect Toolkit method scaleTo() in order to optimize the screen size
            Dim scaledRight As Joint = jointRight.ScaleTo(1440, 900, SkeletonMaxX, SkeletonMaxY)
            pointRight.X = scaledRight.Position.X
            pointRight.Y = scaledRight.Position.Y
    
            Debug.Print("Right Hand Found")
            'pointRight = getDisplayPosition(joints(ids(i)))
            setMyCursor(pointRight.X, pointRight.Y)
          ElseIf ids(i).ToString.Equals("HandLeft") Then
            Debug.Print("Left Hand Found")
            Dim jointLeft As Joint = joints(ids(i))
            'Here i used the Code4fun Kinect Toolkit method scaleTo() in order to optimize the screen size
            Dim scaledLeft As Joint = jointLeft.ScaleTo(1440, 900, SkeletonMaxX, SkeletonMaxY)
            Dim pointLeft As Point
            pointLeft.X = scaledLeft.Position.X
            pointLeft.Y = scaledLeft.Position.Y
    
            'Dim pointLeft As Point = getDisplayPosition(joints(ids(i)))
            'Here we need the coordinates of the right hand in order to know where to 
            'perform the click. The coordinated of the left hand are needed in order 
            'to retrieve if a click should be performed
            performClick(pointRight.X, pointRight.Y, pointLeft.X, pointLeft.Y)
          End If
        Next i
    
      End Function
    


    Here is the extended setMyCursor() method which uses a timer to perform a left click.

     

      Dim firstPositionX As Integer = 0
      Dim firstPositionY As Integer = 0
      Dim timeStart As Date = Now
      Dim setPosition As Boolean = True
      Private Sub setMyCursor(ByVal X As Integer, ByVal Y As Integer)
        'Here i set the initial position to define the area in 
        'which the hand can be moved in order to accept a click.
        If setPosition Then
          firstPositionX = X
          firstPositionY = Y
          setPosition = False
        End If
    
        'Here i check each point to see if its in the limit. If not, then i restart the counter.
        'In this case i allow 200px difference in width and height. Here you can adjust the area to your needs.
        If Math.Abs(X - firstPositionX) > 200 Or Math.Abs(Y - firstPositionY) > 200 Then
          firstPositionX = X
          firstPositionY = Y
          timeStart = Now
        End If
    
        'Here i simply set the counter
        SetCursorPos(X, Y)
    
        'After the time (in this case 1.5 seconds) is reached i perform a left click. 
        'Remeber, the time can be reached only if the if case above has not been entered
        'for the given time. Here you can adjust the time to your needs.
        If (Now - timeStart).TotalSeconds > 1.5 Then
          mouse_event(MOUSEEVENTF_LEFTDOWN, 0, 0, 0, 0)
          mouse_event(MOUSEEVENTF_LEFTUP, 0, 0, 0, 0)
          timeStart = Now
          setPosition = True
        End If
      End Sub
    


    The left click with the right hand is much better when cognition aspects are considered. Adjust the timer to your needs and you will see that its much 

    easier this way. I hope you like it. 

    Friday, July 01, 2011 10:25 AM
  • It's sad that I can't test it yet!

    microsoft! you must sell the kinect power supplyer even with the xbox360 kinect boundle!

    Friday, July 01, 2011 2:01 PM
  • @rcitaliano

     did you manage to find/order the power supply?

    you can also check amazon.de and ebay.de . Several products are shipped in many EU Countries.

     

    Friday, July 01, 2011 2:33 PM
  • i prefer to buy it at the store =)

    but I think that I will have to get it from ebay =/

    Friday, July 01, 2011 2:37 PM
  • Hello,

    I implemented mouse clicking the same way as the selecting is implemented on Kinect Adventure.
    If you don't move your right hand from a certain location in some amount of time, It will register it as a click.

    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    using System.Windows;
    using System.Windows.Controls;
    using System.Windows.Data;
    using System.Windows.Documents;
    using System.Windows.Input;
    using System.Windows.Media;
    using System.Windows.Media.Imaging;
    using System.Windows.Navigation;
    using System.Windows.Shapes;
    using Microsoft.Research.Kinect.Nui;
    using Coding4Fun.Kinect.Wpf;
    using System.Runtime.InteropServices;
    using System.Windows.Forms;
    
    namespace KinectTest
    {
     public class MouseHook
     {
     [DllImport("user32.dll")]
     private static extern void mouse_event(UInt32 dwFlags, UInt32 dx, UInt32 dy, UInt32 dwData, IntPtr dwExtraInfo);
     private const UInt32 MOUSEEVENTF_LEFTDOWN = 0x0002;
     private const UInt32 MOUSEEVENTF_LEFTUP = 0x0004;
    
     internal static void SendClick()
     {
      SendDown();
      SendUp();
     }
    
      internal static void SendUp()
        {
          mouse_event(MOUSEEVENTF_LEFTUP, 0, 0, 0, new System.IntPtr());
        }
        internal static void MoveMouse(Point p)
        {
          System.Windows.Forms.Cursor.Position = new System.Drawing.Point((int)p.X, (int)p.Y);
        }
    
    
        internal static void SendDown()
        {
          mouse_event(MOUSEEVENTF_LEFTDOWN, 0, 0, 0, new System.IntPtr());
        }
    }
    }
    
    namespace KinectTest
    {
     /// <summary>
     /// Interaction logic for MainWindow.xaml
     /// </summary>
     public partial class MainWindow : Window
     {
     DateTime lastClick;
     int lastX = 0;
     int lastY = 0;
     int threshold = 20;
     int actionTime = 1;
     int screenWidth = 2560;
     int screenHeight = 1680;
     public MainWindow()
     {
      InitializeComponent();
     }
    
     //Kinect Runtime
     Runtime nui = new Runtime();
     
     private void Window_Loaded(object sender, RoutedEventArgs e)
     {
      //nui.Initialize(RuntimeOptions.UseColor);
      
      //Initialize to do skeletal tracking
      nui.Initialize(RuntimeOptions.UseSkeletalTracking);
      
      #region TransformSmooth
      //Must set to true and set after call to Initialize
      nui.SkeletonEngine.TransformSmooth = true;
    
      //Use to transform and reduce jitter
      var parameters = new TransformSmoothParameters
      {
      Smoothing = 0.75f,
      Correction = 0.1f,
      Prediction = 0.1f,
      JitterRadius = 0.05f,
      MaxDeviationRadius = 0.06f
      };
    
      nui.SkeletonEngine.SmoothParameters = parameters; 
    
      #endregion
    
      //add event to receive skeleton data
      lastClick = DateTime.Now;
      nui.SkeletonFrameReady += new EventHandler<SkeletonFrameReadyEventArgs>(nui_SkeletonFrameReady);
      
     }
    
     void nui_SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
     {
      SkeletonFrame allSkeletons = e.SkeletonFrame;
    
      foreach (SkeletonData s in allSkeletons.Skeletons)
      {
      
    
    
      if (s.TrackingState == SkeletonTrackingState.Tracked)
      {
       var scaledHandRight = s.Joints[JointID.HandRight].ScaleTo(screenWidth, screenHeight, 0.5f, 0.5f);
       int xCoordinate = (int)scaledHandRight.Position.X;
       int yCoordinate = (int)scaledHandRight.Position.Y;
       if (lastY == 0)
       {
       lastX = xCoordinate;
       lastY = yCoordinate;
       }
       if (Math.Abs(lastX - xCoordinate) < threshold && Math.Abs(lastY - yCoordinate) < threshold)
       {
       if (Math.Abs(lastClick.Subtract(DateTime.Now).TotalSeconds) > 1){
        MouseHook.SendClick();
       }
       }
       MouseHook.MoveMouse(new Point(scaledHandRight.Position.X, scaledHandRight.Position.Y));
      }
      }
     }
    
     private void Window_Closed(object sender, EventArgs e)
     {
      //Cleanup
      nui.Uninitialize();
     }
     }
    }
    
    

     
    There's no place without shadow.

    Saturday, July 02, 2011 2:25 PM
  • i liked it.

    İ dont know C#. Can you publish all source file and/or binary file?

    Saturday, July 02, 2011 3:49 PM
  • @Teemu Tapanila

     

    Hi, nice code. Its the same approach with the second version of code that i sent above. I hadn't the time yet to test it. I am curious to see how good it works with your selected threshold, it seams pretty low. I used a threshold 10 times bigger :)

    Its cool to see codes for these kinds of concepts. Keep going :)

    Saturday, July 02, 2011 4:07 PM
  • @Teemu Tapanila

    Good job! =)

    I already ordered my kinect power supply from ebay... as soon as I get I'll start to first test the code that you and @giasaste posted =) and then trying to implement the whole conecpt (with right clicking heheheh, and scrolling with the mouse wheel) =)

    Sunday, July 03, 2011 1:23 AM
  • Hi Teemu,

     

    Thanks for your awesome code....this is what i am searching for...But where to paste this code in the class file.?

    where is the SendDown()and Sendup() method?

    Please give me step by step procedure in detail?

     

    Regards

    Subash

    Monday, July 04, 2011 9:01 AM
  • hi, i combined Teemus code with the Microsoft example, you might want to see how I implemented it:

    Source: http://dropfiles.buto.ch/27-2011/projectNephalem.rar

    Executable only: http://dropfiles.buto.ch/27-2011/Kinect Mouse Control.rar

    also, here is the full MouseHook class he possibly used for this:

    public class MouseHook
      {
        [DllImport("user32.dll")]
        private static extern void mouse_event(UInt32 dwFlags, UInt32 dx, UInt32 dy, UInt32 dwData, IntPtr dwExtraInfo);
        private const UInt32 MOUSEEVENTF_LEFTDOWN = 0x0002;
        private const UInt32 MOUSEEVENTF_LEFTUP = 0x0004;
    
        internal static void SendClick()
        {
          SendDown();
          SendUp();
        }
    
        internal static void SendUp()
        {
          mouse_event(MOUSEEVENTF_LEFTUP, 0, 0, 0, new System.IntPtr());
        }
        internal static void SendDown()
        {
          mouse_event(MOUSEEVENTF_LEFTDOWN, 0, 0, 0, new System.IntPtr());
        }
    
        internal static void MoveMouse(System.Drawing.Point p)
        {
          System.Windows.Forms.Cursor.Position = new System.Drawing.Point((int)p.X, (int)p.Y);
        }
      }
    


    Monday, July 04, 2011 3:27 PM
  • Any chance you could make a 1-handed kinect mouse.  Also, the more you can reduce the number of finger movements to perform mouse clicking the better for carpul tunnel syndrome.  I have a few thoughts on how this might work...

    1. Lift hand up activates mouse mode, relax hand down deactivates.  this will allow the user to use other inputs at the same time (like the keyboard) without moving the mouse around accidentally.

    2. In mouse mode, X-Y movements of the hand move the mouse in on the screen.

    3. Z movements of the hand would increase and decrease mouse sensitivity.  Normal mouse mode would be the hand moving in X-Y at a comfortable distance away from the chest, when the user extends his arm this would decrease mouse sensitivity allowing for more precise control over the mouse.  Maybe have two or three levels of mouse sensitivity that can be controlled by how far the hand is extended.

    4. An intentional freeze (stopping movement) of the hand, for half a second or so, would automatically click the mouse.  This would greatly reduce fatigue and carpul tunnel syndrome because of significantly fewer finger movements.

    5. The type of click when the hand stops moving would by default be a left click.  There would have to be some other input for right click, double click, ect. maybe the position of the elbow relative to the hand?  or a separate button that is held down?  I'm not sure yet.

    6. Final thought: maybe the user could hold some kind of object - like a pen with a ball at the end of it.  Something that the kinect would easily recognize.  This object would take out calibration issues around different sized hands and would result in much more accurate mousing.

    Wednesday, July 06, 2011 4:37 PM
  • Hi @sef193

    your toughts are really welcome! I will try to see how to put that inside the concept!

    But... i was thinking too that "Kinect mouse" is good, but I think too that we have to change the concept that we use to interact with the computers.

    I'm trying to figure out another concept that would become a middleware for all kinds of software (microsoft compatibility first =) ).

    Wednesday, July 06, 2011 10:56 PM
  • @Gisaste

    Thanks. I'm coding a larger example about how to use the Kinect SDK and this was a small part of the prototype code.
    I will publish the full source + story about it once I get it done.

    The threshold is very low because I'm using the scaled values (This makes a huge difference).

    There's no place without shadow.
    Monday, July 11, 2011 3:10 PM
  • hi Raphael Bucher

    I have added some utilities to that class, take a look

    Here we can control the clicking of the left, right, and middle buttons. I will be updating this

      public class MouseHook
      {
        //used for sending the mouse events like
        [DllImport("user32.dll")]
        private static extern void mouse_event(UInt32 dwFlags, UInt32 dx, UInt32 dy, UInt32 dwData, IntPtr dwExtraInfo);
        //constants for the mouse events
        //found at http://msdn.microsoft.com/en-us/library/ms646273(v=vs.85).aspx
        private const UInt32 MOUSEEVENTF_LEFTDOWN  = 0x0002;  //The left button was pressed
        private const UInt32 MOUSEEVENTF_LEFTUP   = 0x0004;  //The left button was released.
        private const UInt32 MOUSEEVENTF_RIGHTDOWN = 0x08;   //The right button was pressed
        private const UInt32 MOUSEEVENTF_RIGHTUP  = 0x10;   //The left button was released.
        private const UInt32 MOUSEEVENTF_MIDDLEDOWN = 0x0020;  //The middle button was pressed
        private const UInt32 MOUSEEVENTF_MIDDLEUP  = 0x0040;  //The middle button was released.
    
        [DllImport("user32.dll")]
        public static extern short GetAsyncKeyState(UInt16 virtualKeyCode);
        //Virtual key codes
        //found at http://msdn.microsoft.com/en-us/library/dd375731(v=VS.85).aspx
        private const UInt16 VK_MBUTTON = 0x04;//middle mouse button
        private const UInt16 VK_LBUTTON = 0x01;//left mouse button
        private const UInt16 VK_RBUTTON = 0x02;//right mouse button
    
    
        /// <summary>
        /// Returns negative when the button is DOWN and 0 when the button is UP
        /// </summary>
        /// <returns></returns>
        private static short GetMiddleButtonState()
        {
          return GetAsyncKeyState(VK_MBUTTON);
        }
        /// <summary>
        /// Returns negative when the button is DOWN and 0 when the button is UP
        /// </summary>
        /// <returns></returns>
        private static short GetRightButtonState()
        {
          return GetAsyncKeyState(VK_RBUTTON);
        }
        /// <summary>
        /// Returns negative when the button is DOWN and 0 when the button is UP
        /// </summary>
        /// <returns></returns>
        private static short GetLeftButtonState()
        { 
          return GetAsyncKeyState(VK_LBUTTON);
        }
        /// <summary>
        /// Self explanatory
        /// </summary>
        public static bool IsMiddleButtonUp
        {
          get
          {
            return (MouseHook.GetMiddleButtonState() == 0);
          }
        }
        /// <summary>
        /// Self explanatory
        /// </summary>
        public static bool IsLeftButtonUp
        {
          get {
            return (MouseHook.GetLeftButtonState() == 0);
          }
        }
        /// <summary>
        /// Self explanatory
        /// </summary>
        public static bool IsRightButtonUp
        {
          get
          {
            return (MouseHook.GetRightButtonState() == 0);
          }
        }
    
        /// <summary>
        /// Sends the Down and Up Events for the Left Mouse Button
        /// </summary>
        public static void SendClick()
        {
          SendDown();
          SendUp();
        }
        /// <summary>
        /// Clicks as a normal user does,
        /// if the mouse button is up the button will be pressed
        /// if the mouse button is down it will be released
        /// </summary>
        public static void SendClickUser()
        {
          if (IsLeftButtonUp)
            SendDown();
          else
            SendUp();
        }
    
        /// <summary>
        /// Sends the Down and Up Events for the Right Mouse Button
        /// </summary>
        public static void SendRightClick()
        {
          SendDownRight();
          SendUpRight();
        }
        /// <summary>
        /// Clicks as a normal user does,
        /// if the mouse button is up the button will be pressed
        /// if the mouse button is down it will be released
        /// </summary>
        public static void SendRightClickUser()
        {
          if (IsRightButtonUp)
            SendDownRight();
          else
            SendUpRight();
        }
    
        /// <summary>
        /// Send the event Up for the Left Mouse Button
        /// </summary>
        public static void SendUp()
        {
          mouse_event(MOUSEEVENTF_LEFTUP, 0, 0, 0, new System.IntPtr());
        }
    
        /// <summary>
        /// Send the event Down for the Left Mouse Button
        /// </summary>
        public static void SendDown()
        {
          mouse_event(MOUSEEVENTF_LEFTDOWN, 0, 0, 0, new System.IntPtr());
        }
    
        /// <summary>
        /// Send the event Up for the Right Mouse Button
        /// </summary>
        public static void SendUpRight()
        {
          mouse_event(MOUSEEVENTF_RIGHTUP, 0, 0, 0, new System.IntPtr());
        }
    
        /// <summary>
        /// Send the event Down for the Right Mouse Button
        /// </summary>
        public static void SendDownRight()
        {
          mouse_event(MOUSEEVENTF_RIGHTDOWN, 0, 0, 0, new System.IntPtr());
        }
    
    
        /// <summary>
        /// Send the event Up for the Middle Mouse Button
        /// </summary>
        public static void SendMiddleUp()
        {
          mouse_event(MOUSEEVENTF_MIDDLEUP, 0, 0, 0, new System.IntPtr());
        }
    
        /// <summary>
        /// Send the event Down for the Middle Mouse Button
        /// </summary>
        public static void SendMiddleDown()
        {
          mouse_event(MOUSEEVENTF_MIDDLEDOWN, 0, 0, 0, new System.IntPtr());
        }
    
        /// <summary>
        /// Moves the mouse cursor to a point
        /// </summary>
        /// <param name="p">Point to where the mouse cursor will be moved</param>
        public static void MoveMouse(System.Drawing.Point p)
        {
          System.Windows.Forms.Cursor.Position = new System.Drawing.Point((int)p.X, (int)p.Y);
        }
      }
    


    Friday, July 15, 2011 10:43 PM
  • Hi,

    I will publish my new code version later when I get it fully working but here's the idea.

    At least when I was using my right hand as a cursor I stood very still.
    So when moving a cursor I'm checking that the body + foots are kinda still.
    Because when you are moving a lot you probably don't want to move the mouse cursor at that moment.

    I will also implement some kind of distance monitoring. So depending on your screen size and your own length it will tell you how far to go.
    So you can move from corner to corner without moving. 

    There's no place without shadow.
    Sunday, July 17, 2011 1:51 PM
  • Any thoughts on how to do this with microsofts multi-mouse sdk? I want to do this with multiple mouse. I want to recoginize each person from the kinect as a mouse and use that person as a mouse in their sample quiz program. It would be easier i think to fool windows into thinking each person is a generic usb mouse from dell, ibm, and then you can move the mouse using the generic driver commands (after windows is fooled into installing it correctly).

    Thursday, July 28, 2011 2:11 PM
  • Hi @The Thinker

    I have never programmed with "multi-mouse sdk",

    but i can antecipate that with 1 Kinect device you can only keep track of 2 Skeletons, i think, so you would be limited to 2 "mouses".

    But... it should be easy to do the multi mouse if 2 skeletons are supported because you just have to redirect the mouse input to the correct "mouse" controller, any interesting link on that SDK?

    Friday, July 29, 2011 10:10 AM
  • Hi, if I run your executable at http://dropfiles.buto.ch/27-2011/Kinect Mouse Control.rar, will this provide the ability for Kinect to act as a generic mouse. I am not a developer, but am looking for a standard way to demonstrate our software uswing Kinece. I have tried some of the other applications out there like:

    Win&I for Kinect and Kinemote, but these are very slow and a one handed approach makes click and drag very hard. I prefer something two handedDave

    Tuesday, September 06, 2011 3:46 PM
  • I did reasearch and the wdk (windows driver kit) forums people suggested WM_INPUT and raw input to an already created instance of a generic mouse might work but if not i have to code the driver in c (it might be objective c yuk!). I have a thread i will post from their for people if interested. But i will use it for my project. I wonder if in the refresh it supports 4 people. I think it might support 4 people now wont it? Im not for sure if they changed that in the refresh release.

    Im sorry i meant multi-point sdk not multimouse.


    • Edited by The Thinker Tuesday, September 06, 2011 5:22 PM
    Tuesday, September 06, 2011 5:20 PM
  • Hello Kay Too,

    this example does not act like a generic mouse, functions like right click or middle mouse or even scroll are not implemented.
    It is only able to steer the mouse with the right hand and send a left mouse click with stretching your left arm.


    Saturday, September 10, 2011 1:53 PM
  • I figured out that theirs a script that lets you created  an emulated generic hid usb device which includes keyboards and mouse and other devices and send events like mouse click and moving that specific mouse to accomplish my multimouse program so if you can modify that script it will work with multiple mice i just need to figure out how to send the inputs for sure to the 4 emualted mouse. Anyone have updates on this?
    • Edited by The Thinker Tuesday, September 20, 2011 11:49 PM
    Tuesday, September 20, 2011 11:49 PM
  • @The Thinker

    why do you need so many mice at the same time? (just curious)

    and... did u mean multitouch instead of multimouse?

    and can you show/link teh script to us?

    Monday, September 26, 2011 2:57 PM
  • I have the original script on my codeplex page: http://kinectmultipoint.codeplex.com. Its on discussions tab just look for the correct discussion talking about dsf.

    You need dsf(device simulation framework), the codeplex kinect mouse project modified for multiple people, and direct x sdk installed to test my sample which im very close to getting done now. Also, wdk forums have dsf and multiple mouse questions i have posted that entail my latest mouse emulation parts.

    So far overall progress is these: very good! I need to only figure out two things on the emulated mouse device side and two things on kinect side to get this working:

    kinect side:

    1.multiple users need to be added.

    2. support for multiple kinects so i can add 20 or more people as emulated mouse (need for classroom scenario at my work which is IT for public school system)

    emulate mouse side:

    1. make sure the input report doesnt need anything else added to it.

    2. add multiple mouse devices and get them recoginizied in multipoint or mouse mischief.

    After the things above are done i can party like 1953 because clickers and smartboards for schools will be out of style.


    i got permission form the guy of this project for the kinect side of code to use his code modfied for multiple users:

    http://kinectmouse.codeplex.com


    • Edited by The Thinker Thursday, October 20, 2011 12:30 AM
    Thursday, October 20, 2011 12:28 AM
  • Anyone think the kinect mouse sample should be used  underneath the windows 8 layers as a service for apps in windows 8 that dont support kinect directly allowing people basic support of mouse movements?


    Also, have native kinect drivers so people can just start programming with kinect in windows 8.
    • Edited by The Thinker Thursday, October 27, 2011 12:56 PM
    Thursday, October 27, 2011 12:55 PM
  • I wonder has the owner of this thread updated his project? or tell us progess?
    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - "Sherlock holmes" "speak softly and carry a big stick" - theodore roosevelt.
    • Edited by The Thinker Wednesday, November 09, 2011 6:24 PM
    Wednesday, November 09, 2011 6:23 PM
  • I wonder has the owner of this thread updated his project? or tell us progess?
    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - "Sherlock holmes" "speak softly and carry a big stick" - theodore roosevelt.
    heheheh currently my project for the kinect mouse is in stanby, In the meaning time im participating in another project based on c++ that I can't reaveal it's business area XD But... I agree with someone that posted somewhere something like this:
    I see many people programming to have the kinect working as a mouse, I think that the UI should be somewhat redesigned so the kinect could be a really new way to interact with the computer.
    So I'm thinking about this new concept and if i come up with something interesting i'll search again the opinions of the ppl from these nice forums!!!
    Wednesday, November 09, 2011 9:27 PM
  • Yeah i got permission to modify the kinect mouse code located here: http://kinectmouse.codeplex.com so mouse movement is just a start to using multiple players with normal programs. To emulate the mouse means that each mouse can have gestures applied to it in turn allowing multiple gestures to each pointer and have a pointer show what gesture is being performed on a computer like a status helping people above too in thier endeavors but the mouse never will really disappear like how you always have a virtual machine or emulator to play your old games like pacman with style. Plus you can apply characteristics to your mouse improving movement and precision in games like fps because a hand unless your really creativity and willing to sit up for hours is not entertaining to anyone i know that plays those games. No one whos a non-xbox or playstation gamer doesn't know what im talking about because you probably use the computer for casual games but i strongly agree the kinect will make a great monority report scenario.
    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - "Sherlock holmes" "speak softly and carry a big stick" - theodore roosevelt.
    • Edited by The Thinker Thursday, November 10, 2011 12:19 AM
    Thursday, November 10, 2011 12:14 AM
  • How can i change my mouse cursor in a HAND, so basically, i think the mouse cursor is to much old Fashioned.

    So i want a open hand With five fingers, and i want this to replace the mouse cursor so it looks more nice.

    Thx for the help.

     

    Letoir

    Thursday, November 10, 2011 3:19 PM
  • You can change the default mouse cursor to a different one in any app but i have since lost where the docs for that were. I know its possible though to get rid of the default ugly mouse pointer. I will post back when i remember how to do that.

    edit: heres one way thats limited to the form its on: http://www.pcreview.co.uk/forums/change-mouse-cursor-vbulletin-net-t1340392.html


    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - "Sherlock holmes" "speak softly and carry a big stick" - theodore roosevelt.

    • Edited by The Thinker Thursday, November 10, 2011 4:45 PM
    Thursday, November 10, 2011 4:43 PM
  • Hi,
    I am trying to modify the Kinect Mouse example to something done above, when the mouse stays in a certain area for a specific period of time, the native click event is to be fired. 
    The problem is that I do not get the expected results, since I get random clicking most times. I tried to check with breakpoints etc. Most often, when my hand is not visible to Kinect, the cursor goes to the corner of the screen and starts clicking. This is most probably because Math.Abs(lastX - cursorX) < threshold sets to true.
    I have tried changing the threshold values to 200, but it fires a click st the start, and afterwards I am not get expected left clicks, when I hold the cursor in a certain position for some time. Any suggestions would be greatly appreciated. Here's the code:
    //SkeletonFrameReadyevent
    
        foreach (SkeletonData sd in e.SkeletonFrame.Skeletons)
                    {
                        if (sd.TrackingState == SkeletonTrackingState.Tracked)
                        {
                            // make sure both hands are tracked
                            if (sd.Joints[JointID.HandLeft].TrackingState == JointTrackingState.Tracked &&
                                sd.Joints[JointID.HandRight].TrackingState == JointTrackingState.Tracked)
                            {
                                int cursorX, cursorY;
    
                            // get the left and right hand Joints
                            Joint jointRight = sd.Joints[JointID.HandRight];
                            Joint jointLeft = sd.Joints[JointID.HandLeft];
    
                            // scale those Joints to the primary screen width and height
                            Joint scaledRight = jointRight.ScaleTo((int)SystemParameters.PrimaryScreenWidth, (int)SystemParameters.PrimaryScreenHeight, SkeletonMaxX, SkeletonMaxY);
                            Joint scaledLeft = jointLeft.ScaleTo((int)SystemParameters.PrimaryScreenWidth, (int)SystemParameters.PrimaryScreenHeight, SkeletonMaxX, SkeletonMaxY);
    
                            // figure out the cursor position based on left/right handedness
                           
                                cursorX = (int)scaledRight.Position.X;
                                cursorY = (int)scaledRight.Position.Y;
                            //default false, for mouse move, set to true for mouse click
                            bool leftClick;
    
                            if (lastY == 0)
                            {
                                lastX = cursorX;
                                lastY = cursorY;
                            }
    
                            leftClick = false;
                            if (Math.Abs(lastX - cursorX) < threshold && Math.Abs(lastY - cursorY) < threshold)
                            {
                                if (Math.Abs(lastClick.Subtract(DateTime.Now).TotalSeconds) > 1)
                                {
                                    //Mouse click here
                                    leftClick = true;
                                }
                            }
                            
                            //Mouse click when leftDown is true, else Mousemove
                            NativeMethods.SendMouseInput(cursorX, cursorY, (int)SystemParameters.PrimaryScreenWidth, (int)SystemParameters.PrimaryScreenHeight, leftClick);
                            return;
    
    NativeMethods.cs has the following function:
        public static void SendMouseInput(int positionX, int positionY, int maxX, int maxY, bool leftDown)
        		{
        			if(positionX > int.MaxValue)
        				throw new ArgumentOutOfRangeException("positionX");
        			if(positionY > int.MaxValue)
        				throw new ArgumentOutOfRangeException("positionY");
        
        			Input[] i = new Input[2];
        
        			// move the mouse to the position specified
        			i[0] = new Input();
        			i[0].Type = InputMouse;
        			i[0].MouseInput.X = (positionX * 65535) / maxX;
        			i[0].MouseInput.Y = (positionY * 65535) / maxY;
        			i[0].MouseInput.Flags = MouseEventAbsolute | MouseEventMove;
        
        			// determine if we need to send a mouse down or mouse up event
        			if(leftDown)
        			{
        				i[1] = new Input();
        				i[1].Type = InputMouse;
        				i[1].MouseInput.Flags = MouseEventLeftDown;
                        i[1].MouseInput.Flags |= MouseEventLeftUp;
        			}
        
        			// send it off
        			uint result = SendInput(2, i, Marshal.SizeOf(i[0]));
        			if(result == 0)
        				throw new Win32Exception(Marshal.GetLastWin32Error());
        		}
    
    


    Thursday, February 02, 2012 4:42 AM
  • hi @optimus_prime
    it's really nice to see that other persons are trying to implement this concept evolved by this community.

     

    Anyway... I think that your problem might be on the initialization of your variables.
    Giving a meaningfull value is important to make the code work.

     

    So, if you are not doing it yet, in the initialization, I think that lastX and lastX should be set to Zero and you should remove the:
    if (lastY == 0)
    {
     lastX = cursorX;
     lastY = cursorY;
    }
    
    And maybe the lastClick should be initialized to DateTime.Now.

     

    Another thing to note is performance.
    And i think that leftClick should be a static variable outside that method, actually, since you want performance, if you use static variables instead of creating everytime inside the foreach loop I think that you will gain some performance.
    Thursday, February 02, 2012 9:02 AM
  • Hi rctiliano,

    Thanks for the reply and maintaining the thread.

    I have initialized lastClick = DateTime.Now in the Window_loading event in my wpf. I have tried removing that if statement you mentioned, though I am not sure, how the lastX and lastY woud be inviitlized after removing this in the SkeletonFrameReadyEvent.

     

    The mouse moves with the hand, but it does not click when held in a certain area for any time. I have tried increasing the threshold values too (to 200 also)

    My full SkeletonFrameReadyEvent function: http://pastebin.com/P0GPiXJR

    Native methods class: http://pastebin.com/bbXthcna

    Thursday, February 02, 2012 12:20 PM
  • Hi rctiliano,

    Thanks for the reply and maintaining the thread.

    I have initialized lastClick = DateTime.Now in the Window_loading event in my wpf. I have tried removing that if statement you mentioned, though I am not sure, how the lastX and lastY woud be inviitlized after removing this in the SkeletonFrameReadyEvent.

     

    The mouse moves with the hand, but it does not click when held in a certain area for any time. I have tried increasing the threshold values too (to 200 also)

    My full SkeletonFrameReadyEvent function: http://pastebin.com/P0GPiXJR

    Native methods class: http://pastebin.com/bbXthcna


    hahahahah... sorry... you are right. I forgot to mention that you have to put that code right before the SendMouseInput call:

     leftClick = false;
     if (Math.Abs(lastX - cursorX) < threshold && Math.Abs(lastY - cursorY) < threshold)
     {
         if (Math.Abs(lastClick.Subtract(DateTime.Now).TotalSeconds) > 1)
          {
               //Mouse click here
                leftClick = true;
           }
     }
    
    lastX = cursorX;
    lastY = cursorY;
                            
      //Mouse click when leftDown is true, else Mousemove
     NativeMethods.SendMouseInput(cursorX, cursorY, (int)SystemParameters.PrimaryScreenWidth, (int)SystemParameters.PrimaryScreenHeight, leftClick);
    
    

    try it like this

    because you want to "update" those "last" values, only when they will actually become the last ones, and IMO it's after you made your calculations.

    So, as it was b4 you passed the pastebin, you were using always the current values, because you were updating them BEFORE the calculations.

     

    Thursday, February 02, 2012 2:26 PM
  • My work on my project is promising but anyone know of a way to execute vbscript code using any 64-bit scripting engine from microsoft that i can install? Because if i could execute vbscript code directly in my app then i can not have to rewrite necessarily to port into vb.net and that would become a little more stable.

    Yes i have heard of msscript.ocx (Microsoft script control) and mshom.ocx (Microsoft script host object) which are in 32-bit which is my situation. Can i compile the app wholly on 32-bit computer using the whole code from my wsf file and then just use that executable on my 64-bit computer with its dll files?

    Its a multiuser version of the kinect mouse sample sample included for kinect sdk which was posted with other samples on channel 9. I had to create an emulator for this task but as the wdk quote: "we dont provide managed samples for the dsf(device simulation framework) or support for them" it cant be coded in managed code (might have gotten the support for them part wrong but remembering the email from support team).

     I plan on these programs getting support for my app to work in them:

    1. mouse mischief

    2. multipoint mouse sdk not multipoint server.

    3. study island (educational website that has clicker support) which i will get into contact with after i support the above programs.
    4. Join the project and help and you can add support for your own program.

    Just take the TestGenericHid.wsf script and run on any computer with dsf runtime installed and the test mouse device after doing  simulated device manager install will move mouse a little but will be quick like 5 seconds and you wont notice it happened.

    TestGenericHid.wsf vbscript is just a minor change to the one included with the dsf runtime.


    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - "Sherlock holmes" "speak softly and carry a big stick" - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda




    • Edited by The Thinker Sunday, February 05, 2012 12:59 AM
    Sunday, February 05, 2012 12:50 AM
  • Hi @The Thinker,

    IMO you should separate the things out, making your program in 2 parts:

    1) get's the input from many users using one or more kinect devices

    2) use the input, obtained in part1, in another DLL that doesn't use the kinect SDK

    So doing this i think that you already solved the part 1 and now it would be easier to do the second part.

     

    Monday, February 06, 2012 8:28 AM
  • Yes if you  or someone help me out the vbscript code works except i need it changed for multiple devices then it could be converted to another language for execution inside of another program so im so close but it really needs to be converted before i move on anyone on here up to it?

    Also, if you help me with the emulator conversion code it can be used for emulation of any device by changing the report descriptors and input report sent to it.

    Anyone wanting to help i have actually tested it out and the script is compatible on 32-bit and 64-bit.

    Here is project page if anyone wanting to help:

    http://kinectmultipoint.codeplex.com

    If anyone knows c++ closely and you convert the code into a project that can be compiled as a dll and posted it you can modify for your device emulator on the pc if you wish. The wdk guys would gladly help you convert it to c++ but as far as i know powershell is second be alterative unless someone comes up with something in managed code.

     


    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - "Sherlock holmes" "speak softly and carry a big stick" - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda
    • Edited by The Thinker Monday, February 06, 2012 5:29 PM
    Monday, February 06, 2012 5:28 PM
  • Think you can help me out the latest version download for vb.net works after a change somewhere to dsfdevice and adding in a timer till the device driver gets done installing because i tryed it on 32-bit and it worked!

    The only things that need to be done know are:

    1. change so it will create multiple emulated mouse devices.

    2. Add support for mouse mischief powerpoint presentations

    3. Add support for multipoint mouse sdk with kinect

    4. kinect support for study island if they let me

    5. add a discussion if you wish support for any other program and i can help you out.


    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - "Sherlock holmes" "speak softly and carry a big stick" - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda


    • Edited by The Thinker Tuesday, February 07, 2012 7:10 PM
    Tuesday, February 07, 2012 7:09 PM

  •  leftClick = false;
     if (Math.Abs(lastX - cursorX) < threshold && Math.Abs(lastY - cursorY) < threshold)
     {
         if (Math.Abs(lastClick.Subtract(DateTime.Now).TotalSeconds) > 1)
          {
               //Mouse click here
                leftClick = true;
           }
     }
    
    lastX = cursorX;
    lastY = cursorY;
                            
      //Mouse click when leftDown is true, else Mousemove
     NativeMethods.SendMouseInput(cursorX, cursorY, (int)SystemParameters.PrimaryScreenWidth, (int)SystemParameters.PrimaryScreenHeight, leftClick);
    
    

    try it like this

    because you want to "update" those "last" values, only when they will actually become the last ones, and IMO it's after you made your calculations.

    So, as it was b4 you passed the pastebin, you were using always the current values, because you were updating them BEFORE the calculations.

     

    Shouldn't there be another lastClick = DateTime.Now inside this if condition, when the leftClick is set to true?
    Wednesday, February 08, 2012 11:20 AM
  • Shouldn't there be another lastClick = DateTime.Now inside this if condition, when the leftClick is set to true?

    hi @optimus_prime (really, i laugh everytime i type your nickname),

    you are right, you can use that "leftClick" variable to update the "lastClick", I didn't typed it because i tought that you were already using the "leftClick" somewhere to update "lastClick".

    Wednesday, February 08, 2012 2:47 PM
  • @rcitaliano,

    I still think the approach to this 'Kinect Adventure Type' clicking is not correct. I'll try to reword the whole thing:

    Application starts and on Window_load lastClick is given some initial DateTime.

    I hold my right hand without any action and let 3 seconds pass (I am assuming 3 seconds is the threshold time for clicking here)

    I now show my hands to take the mouse cursor somewhere, and hold it stationary. So, it will enter the if condition 

    if (Math.Abs(lastX - cursorX) < threshold && Math.Abs(lastY - cursorY) < threshold)

    Once, it enters this if condition, 3 seconds have already passed, and so it will not wait thrree seconds for the click to happen but starighway will Send a click event.

    There should be some other approach to this 'wait and click'. Any suggestions?


    Friday, February 10, 2012 6:26 AM
  • Hi earth saver optimus_prime  =)

    actually the "kinect Adventure type" clicking was not my suggestion on the kinect mouse concept. But many implemented it because they liked. Check the first post to see the concept i suggested first.

    try this, i added an "else" that should overcome your problem.

    leftClick = false;
     if (Math.Abs(lastX - cursorX) < threshold && Math.Abs(lastY - cursorY) < threshold)
     {
         if (Math.Abs(lastClick.Subtract(DateTime.Now).TotalSeconds) > 1)
          {
               //Mouse click here
                leftClick = true;
           }
     }
    else
    {
     lastClick = DateTime.Now;
     leftClick = false;
    }
    

    I actually implemented the kinect mouse concept and I was able to click, double click and right click. But I somehow lost the code in my HD (I'll do a complete search for it).

    An interesting feature that I implemented was suggested by some user I can't recall (maybe take a look up in posts), The mouse will only move if you're hand is at a certain distance or closer to the kinect. So the program would only change the mouse location if you're hand was inside that range and you could "stop" the mouse imput when you just pushed your hand back. This is a little bit more complex than i tought but i could make it work with a good precision. I will try to write it down in a precise way so everyone can try to implement it.

    I used the Shoudler Joint to check if my hand was over it in Y, if yes it would use the method SendClickUser() (that is in the Mouse Hook Class ) to make the interaction with the mouse button, and if the hand was at a certain distance in Y from my Shoulder it would use again the "SendClickUser()" to finally get the whole Click Event Done ( that is Down and Up).

    Friday, February 10, 2012 8:51 AM
  • Hi rcitaliano,

    Thanks for the reply. Yes, I got something likethis working for me. The problem that I facing is about the memory consumption as you earlier pointed out.

    I have three different operations to do in the SkeletonFrameReady event, for which I fire three functions on this event:

    SkeletonFrameReady += Operation1;

    SkeletonFrameReady += Operation2;

    SkeletonFrameReady += Operation3;

    Inside my application, operation1 and operation2 are working fine. Operation3 which is the mouse tracking and clicking is not a very good experience, since the mouse is not continously tracked but is very choppy.

    Here is the whole of my MousePointerTracking code:

    private void SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e) { foreach (SkeletonData sd in e.SkeletonFrame.Skeletons) { Joint waistpoint = sd.Joints[JointID.HipCenter]; //double waistpoint = sd.Joints[JointID.HipCenter].Position.Z - 0.2; _trackingswitch = ((sd.Joints[JointID.HandRight].Position.Z < (waistpoint.Position.Z - 0.3)) && sd.Joints[JointID.HandRight].Position.Y > waistpoint.Position.Y) ? true : false; if (_trackingswitch) { if (sd.TrackingState == SkeletonTrackingState.Tracked) { // make sure both hands are tracked if (sd.Joints[JointID.HandLeft].TrackingState == JointTrackingState.Tracked && sd.Joints[JointID.HandRight].TrackingState == JointTrackingState.Tracked) { int cursorX, cursorY; // get the left and right hand Joints Joint jointRight = sd.Joints[JointID.HandRight]; Joint jointLeft = sd.Joints[JointID.HandLeft]; // scale those Joints to the primary screen width and height Joint scaledRight = jointRight.ScaleTo((int)SystemParameters.PrimaryScreenWidth, (int)SystemParameters.PrimaryScreenHeight, SkeletonMaxX, SkeletonMaxY); Joint scaledLeft = jointLeft.ScaleTo((int)SystemParameters.PrimaryScreenWidth, (int)SystemParameters.PrimaryScreenHeight, SkeletonMaxX, SkeletonMaxY); // figure out the cursor position based on left/right handedness cursorX = (int)scaledRight.Position.X; cursorY = (int)scaledRight.Position.Y; leftClick = false; if (Math.Abs(lastX - cursorX) < threshold && Math.Abs(lastY - cursorY) < threshold) { if (Math.Abs(lastClick.Subtract(DateTime.Now).TotalSeconds) > 3) { //Mouse click here leftClick = true; } } else { lastClick = DateTime.Now; leftClick = false; } lastX = cursorX; lastY = cursorY; //Mouse click when leftDown is true, else Mousemove NativeMethods.SendMouseInput(cursorX, cursorY, (int)SystemParameters.PrimaryScreenWidth,

    (int)SystemParameters.PrimaryScreenHeight, leftClick); } } } } }


    If I try and use the cursor on the screen with the mouse itself, without using the hands, it goes fine. However, when I try ot move the cursor with my hands, it's very choppy as I said. I wonder if this could be optimised or something to get a good experience out of this.

    Thanks for the help this far.

    Friday, February 10, 2012 10:28 AM
  • hi optimus_prime, I'm glad that I'm helping you.

    The following are some tips to increase the performance, and ONLY TO INCREASE performance.

    you now said that you added 3 handlers to the SkeletonFrameReady, that would really be the OO way to code it out. But sometimes when i need PERFORMANCE I just sacrifice some OO concepts.

    So in your case I would get the Operation1, Operation2, Operation3 inside one unique handler. So you will only have 1 big loop that does everything.

    you can create static variables to store values like SystemParameters.PrimaryScreenWidth when you initialize the program, or you can just type in (hard coded) the values that you want.

    you are creating the variables JointRight, JointLeft and then only using them to be scaled, you can directly use sd.Joints[JointID.HandRight].ScaleTo(....).

    Another thing that might increase performance a bit is removing these "if"s

     if (_trackingswitch)
                    {
                        if (sd.TrackingState == SkeletonTrackingState.Tracked)
                        {
                            // make sure both hands are tracked
                            if (sd.Joints[JointID.HandLeft].TrackingState == JointTrackingState.Tracked &&
                                sd.Joints[JointID.HandRight].TrackingState == JointTrackingState.Tracked)
                            {
    
    I know they are important... but if you want some performance you can sacrifice them ^^ doing this would be desperato tho =).

    But i would also check the enviroment:

    1. Is your computer fast enough? (i used a Core2Duo p8600 and done fine)
    2. Are you using many USB devices? (kinect needs the USB controller to transfer the data you need so in extreme cases if you want to gain that little performance you can remove all the USB devices and use the PS/2 ports if you have them, or use the notebook's mouse pad and keyboard)
    3. Are you running anything else while debugging/running your kinect application? (visual studio is "heavy" so if you can have only that running would be cool)
    4. You can compile the kinect application, close everything (including Visual studio) and then just clicking on the ".exe" that you generated will start the application.
    Friday, February 10, 2012 2:54 PM
    •   Is your computer fast enough? (i used a Core2Duo p8600 and done fine)
    • Are you using many USB devices? (kinect needs the USB controller to transfer the data you need so in extreme cases if you want to gain that little performance you can remove all the USB devices and use the PS/2 ports if you have them, or use the notebook's mouse pad and keyboard)
    • Are you running anything else while debugging/running your kinect application? (visual studio is "heavy" so if you can have only that running would be cool)
    • You can compile the kinect application, close everything (including Visual studio) and then just clicking on the ".exe" that you generated will start the application

    Dont forget that windbg program included with the wdk (windows driver kit) can help with getting more information about a problem when your computer bluescreens on the kinect device driver or when running a app because you can run the anayalze command to help show the root cause of error instead of getting something else using normal visual studio debugged which cant read driver dump files.

    This can help pinpoint performance issues with code also but is a little extreme but will help when your app bluescreens and you need more specific answers.


    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - "Sherlock holmes" "speak softly and carry a big stick" - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda


    • Edited by The Thinker Friday, February 10, 2012 7:36 PM
    Friday, February 10, 2012 7:36 PM
  • Hi All,

    I am trying to implement this to use multiple monitors and seem to be struggling... I am new to the whole Kinect experience so perhaps it's just my lack of understanding but I would have thought it would have been just a matter of changing the scaledRight and scaledLeft System.Parameters.PrimaryScreenWidth to System.Parameters.VirtualScreenWidth, but that doesn't seem to work...

    Any suggestions on where I might be wrong?

    Regards

    Dean

    Sunday, February 12, 2012 10:03 PM
  • Hi DeanHarry,

    Actually i've never done anything for multiple screen...

    you can check if the mouse hook class can help you with that.

    I would try the following:

    1. try to start with fixed values for width and height
    2. if the fixed values don't work, I would try to get the information about all the screens availables to make my calculations, and to know where the mouse actually is
    3. try to set the same resolution on every monitor that you have and test the program.
    Sunday, February 12, 2012 11:03 PM
  • I would ask in the wdk forum and then use the method they suggest as someone asked about multiple screens before their.

    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - "Sherlock holmes" "speak softly and carry a big stick" - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda

    Sunday, February 12, 2012 11:31 PM
  • I would ask in the wdk forum and then use the method they suggest as someone asked about multiple screens before their.

    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - "Sherlock holmes" "speak softly and carry a big stick" - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda

    Hi Thinker,

    Do you mean the Windows Dev Forum? I had a look in there and there is little of use, people are talking about touch screens and multi-monitor but it seems that no one has worked it our as yet...

    I can't see why it should be so hard, If I look at my mouse position X goes from -1920 through to 1920 from extreme left to extreme right.

    The problem I see is in the ScaleTo and setting the width, it assumes that there is only 1 monitor and will only go from 0 to width, which in my case is 3840. 

    What it needs I guess is to be able to set the number of monitors so that if the value is 2 then the starting X position would be PrimaryScreenWidth * -1 and not 0.

    Might make the suggestion fo the Coding4Fun guys :)

    Dean


    • Edited by DeanHarry Monday, February 13, 2012 10:10 AM
    Monday, February 13, 2012 10:06 AM
  • Heres a thread somewhere else where its solved with multiple screens but it seems multiple screens needs to be played with to get working:

    http://www.aspnet-answers.com/microsoft/VB-NET/35137535/screen-cap-multi-monitor-problem.aspx


    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - "Sherlock holmes" "speak softly and carry a big stick" - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda

    Monday, February 13, 2012 8:32 PM
  • This actually turned out to be really quite simple... well in my case anyway :)

    Just set your monitor orientation so that your primary monitor is on the left and secondary is on the right! It kinda of made a lot of sense after thinking about it...

    Thanks to Brian at MS for the insight

    Dean

    Wednesday, February 15, 2012 10:25 AM
  • Hey DeanHarry,

    I think you should create a thread answering your own question, it would make it easier to be found by others.

    Wednesday, February 15, 2012 4:02 PM
  • rcitaliano I would like to use your kinectmouse.codeplex.com code if your the owner in my own project but Im going to create a template for multiple  players also when done with the project so people can edit/use it. ALso, if you could help me stablize my code and better response time that would help me a lot because i just need to identify the actual player now and evert other part is done.

    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - "Sherlock holmes" "speak softly and carry a big stick" - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda

    Friday, March 09, 2012 8:00 PM
  • Hi The Thinker,

    kinectmouse.codeplex.com is not mine =)

    but If it's in my knowledge I would gladly help you with your project.

    Friday, March 09, 2012 11:56 PM
  • Any help you can give on switching between multiple players with kinect code would help. I have code to see how many users passive and active i have but I need to figure out how to identify the active person out of the total players on the active screen in code but I have the recommended release on my project page as kinect code source which im keeping seperate from the actual emulator till im ready for it.

    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - "Sherlock holmes" "speak softly and carry a big stick" - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda

    Monday, March 12, 2012 7:52 PM
  • @rcitaliano & @Teemu Tapanila

    Hello,

    I'm currently working in a project which involves moving a remote device to the right, left, up, and down according to the movements of the right hand using kinect.

    I'm very new with programming and I have some issues getting the MouseCursor examples to run in my computer. I'm now using the SDK v1.0 and C# to program.

    I'd really like to implement the concept of time and to be able to set a range of movements so that the program doesn't react to anything but a swipe of the hand to the right, left, up, or down.

    My idea is very basic (as I told you, I'm just a beginner) but maybe you could help me make it better:

    private void ProcessGestures(Joint center, Joint rightHand, Joint leftHand, Joint head)
            {
                if (leftHand.Position.Y > head.Position.Y && leftHand.Position.X < head.Position.X && FinishMovement == false)
                {
                    if (rightHand.Position.X > center.Position.X + 0.40 && rightHand.Position.Y < center.Position.Y + 0.10 && rightHand.Position.Y > center.Position.Y - 0.05)
                    {
                          active = true;
                          System.Windows.Forms.SendKeys.SendWait("{Right}");
                    }
    
                    if (rightHand.Position.X < center.Position.X && rightHand.Position.Y < center.Position.Y + 0.10 && rightHand.Position.Y > center.Position.Y - 0.05)
                    {
                           active = true;
                           System.Windows.Forms.SendKeys.SendWait("{Left}");
                    }
    
                    if (rightHand.Position.Y > center.Position.Y + 0.20 && rightHand.Position.X > center.Position.X  && rightHand.Position.X < center.Position.X + 0.20)
                    {
                           active = true;
                           System.Windows.Forms.SendKeys.SendWait("{Up}");
                    }
    
                    if (rightHand.Position.Y < center.Position.Y - 0.10 && rightHand.Position.X > center.Position.X  && rightHand.Position.X < center.Position.X + 0.20)
                    {
                           active = true;
                           System.Windows.Forms.SendKeys.SendWait("{Down}");
                    }
      
                }
    
                if (leftHand.Position.Y > center.Position.Y + 0.05 && rightHand.Position.Y > center.Position.Y + 0.05 && rightHand.Position.X < center.Position.X && leftHand.Position.X > center.Position.X)
                {
                    FinishMovement = true;
                    active = false;
                }
    
                if (leftHand.Position.Y > head.Position.Y + 0.05 && leftHand.Position.X < head.Position.X +0.05 && leftHand.Position.X > head.Position.X-0.05)
                {
                    FinishMovement = false;
                }
    
            }

    As you can see I haven't implemented any sort of vector handling, etc since I have tried to run some examples but haven't been able to figure it out.

    I appreciate your help.

    Tuesday, March 27, 2012 8:15 AM
  • Im working on a multiple player implementation of kinectmouse called kinectmultipoint which latest source is located here: http://kinectmultipoint.codeplex.com

    Kinect mouse project source is located here:

    http://kinectmouse.codeplex.com go to source tab and they do have v1 sdk code their but it doesnt show up in downloads section unless they updated the downloads section.


    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - "Sherlock holmes" "speak softly and carry a big stick" - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda

    Tuesday, March 27, 2012 12:05 PM
  • @rcitaliano & @Teemu Tapanila

    Hello,

    I'm currently working in a project which involves moving a remote device to the right, left, up, and down according to the movements of the right hand using kinect.

    I'm very new with programming and I have some issues getting the MouseCursor examples to run in my computer. I'm now using the SDK v1.0 and C# to program.

    I'd really like to implement the concept of time and to be able to set a range of movements so that the program doesn't react to anything but a swipe of the hand to the right, left, up, or down.

    My idea is very basic (as I told you, I'm just a beginner) but maybe you could help me make it better:

    private void ProcessGestures(Joint center, Joint rightHand, Joint leftHand, Joint head)
            {
                if (leftHand.Position.Y > head.Position.Y && leftHand.Position.X < head.Position.X && FinishMovement == false)
                {
                    if (rightHand.Position.X > center.Position.X + 0.40 && rightHand.Position.Y < center.Position.Y + 0.10 && rightHand.Position.Y > center.Position.Y - 0.05)
                    {
                          active = true;
                          System.Windows.Forms.SendKeys.SendWait("{Right}");
                    }
    
                    if (rightHand.Position.X < center.Position.X && rightHand.Position.Y < center.Position.Y + 0.10 && rightHand.Position.Y > center.Position.Y - 0.05)
                    {
                           active = true;
                           System.Windows.Forms.SendKeys.SendWait("{Left}");
                    }
    
                    if (rightHand.Position.Y > center.Position.Y + 0.20 && rightHand.Position.X > center.Position.X  && rightHand.Position.X < center.Position.X + 0.20)
                    {
                           active = true;
                           System.Windows.Forms.SendKeys.SendWait("{Up}");
                    }
    
                    if (rightHand.Position.Y < center.Position.Y - 0.10 && rightHand.Position.X > center.Position.X  && rightHand.Position.X < center.Position.X + 0.20)
                    {
                           active = true;
                           System.Windows.Forms.SendKeys.SendWait("{Down}");
                    }
      
                }
    
                if (leftHand.Position.Y > center.Position.Y + 0.05 && rightHand.Position.Y > center.Position.Y + 0.05 && rightHand.Position.X < center.Position.X && leftHand.Position.X > center.Position.X)
                {
                    FinishMovement = true;
                    active = false;
                }
    
                if (leftHand.Position.Y > head.Position.Y + 0.05 && leftHand.Position.X < head.Position.X +0.05 && leftHand.Position.X > head.Position.X-0.05)
                {
                    FinishMovement = false;
                }
    
            }

    As you can see I haven't implemented any sort of vector handling, etc since I have tried to run some examples but haven't been able to figure it out.

    I appreciate your help.

    Hey Eggycoke, I am also on the similar boat.....did u make any progress with any results???
    Monday, April 16, 2012 11:56 PM
  • Ive been looking at kinect toolbox but iam waiting for v1.5 of kinect sdk which looks promising.  link: http://kinecttoolbox.codeplex.com

    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - "Sherlock holmes" "speak softly and carry a big stick" - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda. Blog - http://jefferycarlsonblog.blogspot.com/



    • Edited by The Thinker Tuesday, April 17, 2012 12:05 AM
    Tuesday, April 17, 2012 12:04 AM
  • hi @eggycoke, @ihsmav

    you search up in the thread's post and you'll find a conversation started by the user optimus_prime 

    we discussed about the time behavior and it seem that he got it solved.

    while you read the rest of this, click on the links to find more info, they are all links from www.msdn.microsoft.com
    anyway the concept of it is creating a DateTime variable like this:

    DateTime startingEvent;

    then when you have that "active = true" you update that variable using the current time:

    startingEvent = DateTime.Now;

    and then when you have that "active = false" you make a subtraction like this:

    TimeSpan result = DateTime.Now-startingEvent;

    that resul variable holds the time lapse that occurred between "active = true" and "active = false" and

    and doing result.TotalSeconds will give you the amount of seconds that passed.

    so then you should do something like

    if(result.TotalSeconds >= 3)

    to check if the seconds that passed were 3 or more or some value that you wish.

    so after you done the calculations you needed you have to reset the startingEvent to DateTime.Now again and keep the process going.

    i hope this helps


    • Edited by rcitaliano Thursday, April 19, 2012 7:11 PM
    Thursday, April 19, 2012 7:06 PM
  • I did another implementation using the shoulder and hand to get the position of the mouse.

    Here is a video:

    https://www.dropbox.com/s/hcmc5n61vx0jlpy/EjemploPintar.mp4


    • Edited by EdgarTaor Thursday, June 07, 2012 11:06 PM
    Thursday, June 07, 2012 11:06 PM
  • Hi.

    I am new to this whole kinect developing area. your reply caught my attention, have you figured out a way to do that ?

    i mean , using just 1 hand for movement, scrolling, dragging and also clicking. do let me know. i am really interested!

    Tuesday, February 05, 2013 2:21 PM
  • Hi Tareq,

    can i see the the codes ? i am looking for something that allows 1 hand movement (instead of both hands)

    Thanks

    Tuesday, February 05, 2013 2:22 PM
  • Hi ksna,

    I didn't personally implemented the mouse concept using just one hand, but thinking about it doesn't seem that hard to do.

    You could use the shoulder joint as EdgarTaor suggested, or even you could use fixed values if you wish, but i guess that using the shoulder as the point of reference should be better.

    The hardest thing here is actually creating a concept of how it will work, by concept i mean the gestures that you will need to do to make certain things.

    Thinking fast about it i would suggest:

    1. Main trigger would still be time based.
    2. Click: if your hand is X mm in front of the relative should the action will be a click
    3. Scrolling: if your hand is X mm to the right of your shoulder the action will be scroll
    4. Mouse event down (drag): if your hand is X mm to the left of yout shoulder the action will be just a Mouse event down, and triggering the action again will result in the Mouse event UP and this will end the dragging.

    So as you see the hardest thing will be defining a concept, you can use this if you want, but maybe that's something better to think about

    Hope it helped

    Tuesday, February 05, 2013 2:33 PM
  • Hi Tareq,

    can i see the the codes ? i am looking for something that allows 1 hand movement (instead of both hands)

    Thanks

    along the thread you will see pieces of code,

    and at the start I try to keep all the code contributions listed

    Tuesday, February 05, 2013 2:35 PM
  • I am almost done with my multiple mouse program for emulating multiple mouse devices with Kinect and its source is listed on http://kinectmultipoint.codeplex.com.

    You can add mouse scrolling hardware feature by modifying the vbscript file I use together with the vb.net code. 

    I have a new source code release coming soon. Then the next release will have an installer. 


    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - "Sherlock holmes" "speak softly and carry a big stick" - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda. Blog - http://www.computerprofessions.co.nr


    • Edited by The Thinker Wednesday, February 06, 2013 8:00 PM
    Wednesday, February 06, 2013 7:59 PM
  • hi, i combined Teemus code with the Microsoft example, you might want to see how I implemented it:

    ....

    also, here is the full MouseHook class he possibly used for this:

    public class MouseHook
      {
        [DllImport("user32.dll")]
        private static extern void mouse_event(UInt32 dwFlags, UInt32 dx, UInt32 dy, UInt32 dwData, IntPtr dwExtraInfo);
        private const UInt32 MOUSEEVENTF_LEFTDOWN = 0x0002;
        private const UInt32 MOUSEEVENTF_LEFTUP = 0x0004;
    
        internal static void SendClick()
        {
          SendDown();
          SendUp();
        }
    
        internal static void SendUp()
        {
          mouse_event(MOUSEEVENTF_LEFTUP, 0, 0, 0, new System.IntPtr());
        }
        internal static void SendDown()
        {
          mouse_event(MOUSEEVENTF_LEFTDOWN, 0, 0, 0, new System.IntPtr());
        }
    
        internal static void MoveMouse(System.Drawing.Point p)
        {
          System.Windows.Forms.Cursor.Position = new System.Drawing.Point((int)p.X, (int)p.Y);
        }
      }
    


    hi Raphael, i just downloaded your executable, but Windows 10 tells me it could crash my system. i really need an executable that makes my Kinect as a mouse, i can't write code. can you help me?
    Tuesday, November 24, 2015 7:10 PM
  • hello there,

    I don't know if some1 finally implemented the solution, but you can check out The Thinker's website http://kinectmultipoint.codeplex.com, you might find some executables.

    Wednesday, November 25, 2015 9:10 AM
  • hello there,

    I don't know if some1 finally implemented the solution, but you can check out The Thinker's website http://kinectmultipoint.codeplex.com, you might find some executables.

    thank you!

    but doesn't work!

    maybe there's somethings wrong i do, i don't know: i downloaded the last one, but there's no .exe inside. And i need that.

    Thursday, November 26, 2015 1:30 PM
  • hello there,

    I don't know if some1 finally implemented the solution, but you can check out The Thinker's website http://kinectmultipoint.codeplex.com, you might find some executables.

    thank you!

    but doesn't work!

    maybe there's somethings wrong i do, i don't know: i downloaded the last one, but there's no .exe inside. And i need that.

    it does work and few Kinect projects have an exe but mine should be able to be compiled out of the box. Use the newer versions and not the vbscript ones (I am halfway through the mouse device emulator and it works on windows 8.1 and above). Note: running any projects you find on the internet you need at least the Kinect runtime but to code need the SDK (Xbox 360 Kinect only works if the full SDK is installed). In addition, include the dll or device driver project into your code its the one under beta.

    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - Sherlock Holmes. speak softly and carry a big stick - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda. Blog - http://www.computerprofessions.us

    Thursday, November 26, 2015 1:48 PM
  • hello there,

    I don't know if some1 finally implemented the solution, but you can check out The Thinker's website http://kinectmultipoint.codeplex.com, you might find some executables.

    thank you!

    but doesn't work!

    maybe there's somethings wrong i do, i don't know: i downloaded the last one, but there's no .exe inside. And i need that.

    it does work and few Kinect projects have an exe but mine should be able to be compiled out of the box. Use the newer versions and not the vbscript ones (I am halfway through the mouse device emulator and it works on windows 8.1 and above). Note: running any projects you find on the internet you need at least the Kinect runtime but to code need the SDK (Xbox 360 Kinect only works if the full SDK is installed). In addition, include the dll or device driver project into your code its the one under beta.

    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - Sherlock Holmes. speak softly and carry a big stick - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda. Blog - http://www.computerprofessions.us


    i have a v1 Kinect with adapter. is it the same? and i would like to have the Kinect as a mouse all over Windows (i have 10), not just for an application. how can i do that? i'm desperate!
    Thursday, November 26, 2015 2:01 PM
  • hello there,

    I don't know if some1 finally implemented the solution, but you can check out The Thinker's website http://kinectmultipoint.codeplex.com, you might find some executables.

    thank you!

    but doesn't work!

    maybe there's somethings wrong i do, i don't know: i downloaded the last one, but there's no .exe inside. And i need that.

    it does work and few Kinect projects have an exe but mine should be able to be compiled out of the box. Use the newer versions and not the vbscript ones (I am halfway through the mouse device emulator and it works on windows 8.1 and above). Note: running any projects you find on the internet you need at least the Kinect runtime but to code need the SDK (Xbox 360 Kinect only works if the full SDK is installed). In addition, include the dll or device driver project into your code its the one under beta.

    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - Sherlock Holmes. speak softly and carry a big stick - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda. Blog - http://www.computerprofessions.us


    i have a v1 Kinect with adapter. is it the same? and i would like to have the Kinect as a mouse all over Windows (i have 10), not just for an application. how can i do that? i'm desperate!
    install the Kinect SDK before trying any examples! You need at least the V1(version 1) SDK not beta versions and this maybe the inappropriate forum if you have the new version 2 Kinect.

    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - Sherlock Holmes. speak softly and carry a big stick - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda. Blog - http://www.computerprofessions.us

    Thursday, November 26, 2015 2:14 PM
  • install the Kinect SDK before trying any examples! You need at least the V1(version 1) SDK not beta versions and this maybe the inappropriate forum if you have the new version 2 Kinect.

    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - Sherlock Holmes. speak softly and carry a big stick - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda. Blog - http://www.computerprofessions.us

    i'm sure it's a Kinect v1 and i already installed sdk 2.0, but i also have 1.6 and 1.8: is the 2.0 the good one? how can i try the examples? i'm not a programmer, i need an executable file.
    Thursday, November 26, 2015 3:57 PM
  • install the Kinect SDK before trying any examples! You need at least the V1(version 1) SDK not beta versions and this maybe the inappropriate forum if you have the new version 2 Kinect.

    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - Sherlock Holmes. speak softly and carry a big stick - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda. Blog - http://www.computerprofessions.us

    i'm sure it's a Kinect v1 and i already installed sdk 2.0, but i also have 1.6 and 1.8: is the 2.0 the good one? how can i try the examples? i'm not a programmer, i need an executable file.
    you should only have v1 if you got the code from this forum. Their is a separate forum for V2 SDK. The Kinect code can be compiled and played with but the newest code relies on the device driver for emulating multiple mouse devices. It is buggy but will soon be plain usable.

    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - Sherlock Holmes. speak softly and carry a big stick - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda. Blog - http://www.computerprofessions.us

    Thursday, November 26, 2015 5:07 PM
  • note: the link to http://kinectmouse.codeplex.com should only be version 1 Kinect SDK compatible. I strongly suggest remving all Kinect SDKs except V1•8 if you use anything from this post unless the author specifically says so.

    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - Sherlock Holmes. speak softly and carry a big stick - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda. Blog - http://www.computerprofessions.us

    Thursday, November 26, 2015 5:11 PM
  • note: the link to http://kinectmouse.codeplex.com should only be version 1 Kinect SDK compatible. I strongly suggest remving all Kinect SDKs except V1•8 if you use anything from this post unless the author specifically says so.

    Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth. - Sherlock Holmes. speak softly and carry a big stick - theodore roosevelt. Fear leads to anger, anger leads to hate, hate leads to suffering - Yoda. Blog - http://www.computerprofessions.us


    it's a v1, as i already told. no, i just tryed your solution, and i'll wait for the exe :) thank you, anyway
    Thursday, November 26, 2015 6:59 PM