none
Why does this work so poorly? - replaying mouse position on screen. RRS feed

  • Question

  • Hi all,

    I've tried to make a mouse position playback application (with the help of the interwebs, of course), however, when playing back the mouse position, it's quite random (no so bad that the cursor is flying from one side of the screen to the other, but it's not anywhere near where I recorded it at). When recording the X,Y positions of the cursor, the coordinates add up, so I don't think that's the problem. Here's my code:

    Public Class Form1
        Private Sub Rec_Click(sender As Object, e As EventArgs) Handles Rec.Click
            Playback.Enabled = False
            Record.Enabled = True
        End Sub
    
        Private Sub ply_Click(sender As Object, e As EventArgs) Handles ply.Click
            Record.Enabled = False
            Playback.Enabled = True
            MouseX.SelectedIndex = 0
            MouseY.SelectedIndex = 0
        End Sub
    
        Private Sub Record_Tick(sender As Object, e As EventArgs) Handles Record.Tick
            MyX.Text = Cursor.Position.X
            MyY.Text = Cursor.Position.Y
    
            MouseY.Items.Add(MyX.Text)
            MouseY.SelectedIndex = MouseY.SelectedIndex + 1
    
            MouseX.Items.Add(MyX.Text)
            MouseX.SelectedIndex = MouseX.SelectedIndex + 1
        End Sub
    
        Private Sub Playback_Tick(sender As Object, e As EventArgs) Handles Playback.Tick
            Dim TotalItems As Integer = MouseX.Items.Count - 1
            Dim SelectedIndex As Integer = MouseX.SelectedIndex
    
            If TotalItems = SelectedIndex = False Then
                Cursor.Position = New Point _
                    (MouseX.SelectedItem, MouseY.SelectedItem)
                MouseX.SelectedIndex = MouseX.SelectedIndex + 1
                MouseY.SelectedIndex = MouseY.SelectedIndex + 1
            Else
                Playback.Enabled = False
            End If
        End Sub
    End Class

    The interface is simple, and the form consists of two timers ('Record' and 'Playback')

    Is anyone able to suggest why the playback is so random?


    • Edited by Lucy Qu Saturday, November 4, 2017 7:47 PM Because it's a nice day today.
    Saturday, November 4, 2017 7:46 PM

Answers

  •  This may be due to your screen settings.  Windows Form applications are not Dpi Aware by default and if you are using a screen scaling setting such as 150%,  the windows form methods will not return the correct scaled positions or sizes.  There are fixes for that but,  first i would check these things...

    First i would HIGHLY recommend turning Option Strict on and correct any type conversion errors,  if any.  You can place these lines at the top of your code to turn it on for just this document.  They will help you correct any Type Conversion Errors.

    Option Strict On
    Option Explicit On
    Option Infer OFF
     

     The next thing that stands out right off the bat is that you are assigning the X 'String Type' value to both the MouseX.Items and the MouseY.Items in these lines...

            MouseY.Items.Add(MyX.Text)
            MouseY.SelectedIndex = MouseY.SelectedIndex + 1
    
            MouseX.Items.Add(MyX.Text)
            MouseX.SelectedIndex = MouseX.SelectedIndex + 1

     

     See where that gets you....  8)


    If you say it can`t be done then i`ll try it

    • Edited by IronRazerz Saturday, November 4, 2017 9:00 PM
    • Marked as answer by Lucy Qu Saturday, November 4, 2017 9:12 PM
    Saturday, November 4, 2017 8:19 PM
  •  PS - This line would work but,  is odd....

    If TotalItems = SelectedIndex = False Then


     That should look more like below.  You would not need to check if (TotalItems = SelectedIndex) is equal to True.  Just checking (TotalItems = SelectedIndex) will give the True or False condition.

    If TotalItems = SelectedIndex Then


    If you say it can`t be done then i`ll try it

    • Edited by IronRazerz Saturday, November 4, 2017 8:48 PM
    • Marked as answer by Lucy Qu Saturday, November 4, 2017 9:12 PM
    Saturday, November 4, 2017 8:43 PM
  • Hi

    Suggest you put

    Option Strict On
    Option Explicit On

    at the top of your Form code.

    *

    Because you didn't really supply all the information needed, I have rebuilt using someof your code and someof my own. See if it helps.

    ' Form1 with 2 Timers named 'record' and 'playback'
    ' 2 Buttons named 'rec' and 'ply'
    ' 3 TextBoxes named 'MyX' and 'MyY' and 'Clear'
    Option Strict On
    Option Explicit On
    Public Class Form1
        Dim MousePos As New List(Of Point)
        Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
            Playback.Interval = 100
            Record.Interval = 100
    
            Playback.Enabled = False
            Record.Enabled = False
        End Sub
        Private Sub Rec_Click(sender As Object, e As EventArgs) Handles Rec.Click
            Select Case Rec.Text
                Case "Rec"
                    Playback.Enabled = False
                    Record.Enabled = True
                    Rec.Text = "STOP"
                Case Else
                    Record.Enabled = False
                    Rec.Text = "Rec"
            End Select
        End Sub
        Private Sub ply_Click(sender As Object, e As EventArgs) Handles ply.Click
            Playback.Enabled = True
            Record.Enabled = False
            Rec.Text = "Rec"
        End Sub
        Private Sub Clear_Click(sender As Object, e As EventArgs) Handles Clear.Click
            ply.Text = "Play"
            Rec.Text = "Rec"
            MousePos.Clear()
        End Sub
        Private Sub Record_Tick(sender As Object, e As EventArgs) Handles Record.Tick
            MyX.Text = Cursor.Position.X.ToString
            MyY.Text = Cursor.Position.Y.ToString
            MousePos.Add(Cursor.Position)
        End Sub
        Private Sub Playback_Tick(sender As Object, e As EventArgs) Handles Playback.Tick
            If MousePos.Count < 1 Then Exit Sub
            Static p As Integer = 0
            Cursor.Position = MousePos(p)
            p += 1
            MyX.Text = Cursor.Position.X.ToString
            MyY.Text = Cursor.Position.Y.ToString
            If p = MousePos.Count Then
                Playback.Enabled = False
                Playback.Enabled = False
                p = 0
            End If
        End Sub
    End Class



    Regards Les, Livingston, Scotland

    • Marked as answer by Lucy Qu Saturday, November 4, 2017 9:12 PM
    Saturday, November 4, 2017 8:47 PM

All replies

  •  This may be due to your screen settings.  Windows Form applications are not Dpi Aware by default and if you are using a screen scaling setting such as 150%,  the windows form methods will not return the correct scaled positions or sizes.  There are fixes for that but,  first i would check these things...

    First i would HIGHLY recommend turning Option Strict on and correct any type conversion errors,  if any.  You can place these lines at the top of your code to turn it on for just this document.  They will help you correct any Type Conversion Errors.

    Option Strict On
    Option Explicit On
    Option Infer OFF
     

     The next thing that stands out right off the bat is that you are assigning the X 'String Type' value to both the MouseX.Items and the MouseY.Items in these lines...

            MouseY.Items.Add(MyX.Text)
            MouseY.SelectedIndex = MouseY.SelectedIndex + 1
    
            MouseX.Items.Add(MyX.Text)
            MouseX.SelectedIndex = MouseX.SelectedIndex + 1

     

     See where that gets you....  8)


    If you say it can`t be done then i`ll try it

    • Edited by IronRazerz Saturday, November 4, 2017 9:00 PM
    • Marked as answer by Lucy Qu Saturday, November 4, 2017 9:12 PM
    Saturday, November 4, 2017 8:19 PM
  •  PS - This line would work but,  is odd....

    If TotalItems = SelectedIndex = False Then


     That should look more like below.  You would not need to check if (TotalItems = SelectedIndex) is equal to True.  Just checking (TotalItems = SelectedIndex) will give the True or False condition.

    If TotalItems = SelectedIndex Then


    If you say it can`t be done then i`ll try it

    • Edited by IronRazerz Saturday, November 4, 2017 8:48 PM
    • Marked as answer by Lucy Qu Saturday, November 4, 2017 9:12 PM
    Saturday, November 4, 2017 8:43 PM
  • Hi

    Suggest you put

    Option Strict On
    Option Explicit On

    at the top of your Form code.

    *

    Because you didn't really supply all the information needed, I have rebuilt using someof your code and someof my own. See if it helps.

    ' Form1 with 2 Timers named 'record' and 'playback'
    ' 2 Buttons named 'rec' and 'ply'
    ' 3 TextBoxes named 'MyX' and 'MyY' and 'Clear'
    Option Strict On
    Option Explicit On
    Public Class Form1
        Dim MousePos As New List(Of Point)
        Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
            Playback.Interval = 100
            Record.Interval = 100
    
            Playback.Enabled = False
            Record.Enabled = False
        End Sub
        Private Sub Rec_Click(sender As Object, e As EventArgs) Handles Rec.Click
            Select Case Rec.Text
                Case "Rec"
                    Playback.Enabled = False
                    Record.Enabled = True
                    Rec.Text = "STOP"
                Case Else
                    Record.Enabled = False
                    Rec.Text = "Rec"
            End Select
        End Sub
        Private Sub ply_Click(sender As Object, e As EventArgs) Handles ply.Click
            Playback.Enabled = True
            Record.Enabled = False
            Rec.Text = "Rec"
        End Sub
        Private Sub Clear_Click(sender As Object, e As EventArgs) Handles Clear.Click
            ply.Text = "Play"
            Rec.Text = "Rec"
            MousePos.Clear()
        End Sub
        Private Sub Record_Tick(sender As Object, e As EventArgs) Handles Record.Tick
            MyX.Text = Cursor.Position.X.ToString
            MyY.Text = Cursor.Position.Y.ToString
            MousePos.Add(Cursor.Position)
        End Sub
        Private Sub Playback_Tick(sender As Object, e As EventArgs) Handles Playback.Tick
            If MousePos.Count < 1 Then Exit Sub
            Static p As Integer = 0
            Cursor.Position = MousePos(p)
            p += 1
            MyX.Text = Cursor.Position.X.ToString
            MyY.Text = Cursor.Position.Y.ToString
            If p = MousePos.Count Then
                Playback.Enabled = False
                Playback.Enabled = False
                p = 0
            End If
        End Sub
    End Class



    Regards Les, Livingston, Scotland

    • Marked as answer by Lucy Qu Saturday, November 4, 2017 9:12 PM
    Saturday, November 4, 2017 8:47 PM
  • IronRazerz - Thank you once again! I typically enable Option Strict in the "Compile" tab of "My Project", but because this was a new form, it was disabled, thanks for pointing that out. But the main issue was definitely the huge error in the last code block in your message; oops! Marked as answer and upvoted - thank you!

    leshey - thanks very much, after building on Iron's suggestions, I tried that code too and it works perfectly, probably better than my "fixed code". The clear button's useful too. Gey helpful, cheers fur ye hulp!

    Saturday, November 4, 2017 9:18 PM
  • Just another thing, if I were to want to isolate the mouse activity to the form only (useful if working the application on different resolutions of screen), would I just have to replace "cursor.position" with "picturebox1.position", since, essentially, the cursor class represents the image used to paint the mouse cursor.

    PointToClient mabye?
    • Edited by Lucy Qu Saturday, November 4, 2017 9:26 PM
    Saturday, November 4, 2017 9:24 PM
  •  I am not sure if i understand exactly what you are asking but,  if you just want to record mouse activity that is on a PictureBox in your application,  then it would be most reliable using the MouseDown,  MouseMove, and MouseUp events of the picturbox to record the mouse activity.

     In all of those mouse events you can use the MouseEventArgs that are passed to the event subs in the e parameter to get the mouse position and any mouse buttons that are presses.  For a basic example of getting the mouse buttons or mouse location in a PictureBox...

    Public Class Form1
        Private Sub PictureBox1_MouseDown(sender As Object, e As MouseEventArgs) Handles PictureBox1.MouseDown
            Me.Text = e.Button.ToString & " down at " & e.Location.ToString
        End Sub
    
        Private Sub PictureBox1_MouseMove(sender As Object, e As MouseEventArgs) Handles PictureBox1.MouseMove
            If e.Button = MouseButtons.Left Then
                Me.Text = "Left Down and is at " & e.Location.ToString
            End If
        End Sub
    
        Private Sub PictureBox1_MouseUp(sender As Object, e As MouseEventArgs) Handles PictureBox1.MouseUp
            Me.Text = e.Button.ToString & " up at " & e.Location.ToString
        End Sub
    End Class
    


    If you say it can`t be done then i`ll try it

    Saturday, November 4, 2017 9:43 PM
  • What I meant was, in order to allow mouse activity in multiple forms, even if they're minimized, it might be a good idea to use a picturebox (the image being a cursor) to move around the form. This way I could continue to operate the main mouse and if I were to add click functions to these cursors (actually pictureboxes), they can perform them in multiple copies of my form. I'm already working on it, hopefully it will work.
    Saturday, November 4, 2017 9:54 PM