none
Simulate Click to Minimized Window VB RRS feed

  • Question

  • Hello everyone,

    I need to simulate a mouse click at specific coordinates of a certain minimized application. I am using visual studio, visual basic, and I am not sure how to proceed. I have done a lot of research, but I am not a skilled enough programmer yet to piece together what I have found on the internet. I would be very grateful if someone could show me a simple program to send a click at specific coordinates into a minimized window and maybe explain it so I can understand what is happening and write my own code. Thank you very much, and have a great day.


    Friday, October 13, 2017 6:50 PM

Answers

  • thank you, I'll look into it

    Well if you want to click on a document somewhere which is being displayed within a WebBrowser control then there is code that can perform that. However you would have to know what on the document is supposed to be clicked on such as an HTML documents (see W3 Schools) HTML Links which are "Hyperlinks" located on the document. Also the document may contain I suppose they would be called Hyperlinked images, videos, etc. But you would probably need to be very specific in describing exactly what you desire and possible provide a URL to the WebPage you intend to perform this with so others could view the source code to see if the possibility exists of performing what you want.

    I only come on here once a month really so the delay is my fault.

    And then rather than a mouse click to the HTML document displayed in the WebBrowser control you could possibly set up a Hot Key in some fashion to codewise "auto click" on the link in the WebPage displayed in the WebBrowser control.

    IronRazerz knows about that stuff but since you are unclear on the exact detail necessary for a programmer to assist you then everybody is guessing what you mean by "My apologies (and thank you for the help so far).

    The form is what I want to click on, not any child control. See, I am using visual studio web browser, and so I have the form with the web browser, so I need to click on certain coordinates of said web browser which is built into the form. I think this is what you meant by form vs. control, let me know if it isn't." as a WebBrowser control can be mouse clicked on in various locations as can a WebPage in a WebBrowser control if the WebPage has something clickable but since a WebPage is typically written in HMTL/CSS/Java Script I suppose then others could view the source code of the WebPage if provided a link to it to see which area you desire to initiate a link within the document.

    If none of that seems as what you desire then please explain further.


    La vida loca

    • Marked as answer by Wilkyrl Wednesday, November 1, 2017 7:00 PM
    Wednesday, November 1, 2017 4:40 PM
  • unfortunately, I have only gotten answers on how to click on a window with focus, not a window that is minimized or a background process.

     You have not really answered the questions i asked but,  i will take a wild guess that these really are two separate applications that you made and not just one application with two forms which would be much easier than communicating between separate applications.

     Below is an example of using the COPYDATASTRUCT structure,  the WM_COPYDATA message, and the SendMessage function for doing this using Interprocess Communications (as i mentioned earlier).  There are other methods for interprocess communication so,  this is just one way of doing it which communicates directly between your two applications.  Below is the example code for both applications,  the first one below is the application with the WebBrowser control on the form.

    Imports System.Runtime.InteropServices
    
    Public Class Form1
        Private Const WM_COPYDATA As Integer = &H4A
    
        <StructLayout(LayoutKind.Sequential)>
        Public Structure COPYDATASTRUCT
            Public dwData As IntPtr
            Public cbData As Integer
            Public lpData As IntPtr
        End Structure
    
        Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
            WebBrowser1.ScriptErrorsSuppressed = True
            WebBrowser1.Navigate("https://technet.microsoft.com/en-us/library/bb490996.aspx")
        End Sub
    
        Protected Overrides Sub WndProc(ByRef m As Message)
            MyBase.WndProc(m)
    
            If m.Msg = WM_COPYDATA Then 'check if the WM_COPYDATA message was sent to this application's window
    
                Dim cds As COPYDATASTRUCT = CType(m.GetLParam(GetType(COPYDATASTRUCT)), COPYDATASTRUCT) 'get the COPYDATASTRUCT that was passed to the lParam
    
                Dim Data(cds.cbData - 1) As Byte 'create a byte array of the size that the cbData was set to by the application that sent the message
                Marshal.Copy(cds.lpData, Data, 0, Data.Length) 'copy the byte array from the unmanaged memory pointed to by the lpData member
    
                Dim ElementInnerText As String = System.Text.Encoding.ASCII.GetString(Data) 'Convert the bytes back into a string. This is the text that will identify the html element to click.
    
                'get the html element from the document that is loaded in the WebBrowser and invoke a 'click' on it.
                Dim elm As HtmlElement = (From el As HtmlElement In WebBrowser1.Document.All.Cast(Of HtmlElement) Where el.InnerText = ElementInnerText).FirstOrDefault
                elm.InvokeMember("click")
            End If
        End Sub
    End Class
    
     

     This is the second application which has one button on it that will send the message to the WebBrowser application.

    Imports System.Runtime.InteropServices
    
    Public Class Form1
        Private Const WM_COPYDATA As Integer = &H4A
    
        <DllImport("user32.dll", EntryPoint:="SendMessageW")>
        Private Shared Function SendMessageW(ByVal hWnd As IntPtr, ByVal Msg As UInteger, ByVal wParam As IntPtr, ByRef lParam As COPYDATASTRUCT) As Integer
        End Function
    
        <StructLayout(LayoutKind.Sequential)>
        Private Structure COPYDATASTRUCT
            Public dwData As IntPtr
            Public cbData As Integer
            Public lpData As IntPtr
        End Structure
    
        Private cds As New COPYDATASTRUCT
    
        Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
            Dim p As Process = Process.GetProcessesByName("test click webbrowser app").FirstOrDefault 'get the first process of the application that has the WebBrowser on it
    
            If p IsNot Nothing Then 'make sure the process was found
    
                Dim bts() As Byte = System.Text.Encoding.ASCII.GetBytes("Wiki") 'get a byte array of a string that will identify the html element you want to click in the other app's WebBrowser control
    
                cds.cbData = bts.Count 'set this to the number of byte so that the other app know how many bytes to read
                cds.lpData = Marshal.AllocHGlobal(bts.Length) 'allocate enough unmanaged memory to hold the byte array
                Marshal.Copy(bts, 0, cds.lpData, bts.Length) 'copy the byte array to the allocated memory pointed to by the lpData member of the COPYDATASTRUCT
    
                SendMessageW(p.MainWindowHandle, WM_COPYDATA, Me.Handle, cds) 'send the WM_COPYDATA message to the other application's handle, passing the COPYDATASTRUCT in the lParam
    
                Marshal.FreeHGlobal(cds.lpData) 'now free the unmanaged memory
            Else
                MessageBox.Show("The other application was not found.")
            End If
        End Sub
    End Class
    
     

     Here you can see it working....

     

     You will need to stud the code,  research what you don't understand,  and experiment a little with it until you understand what it does and how it does it.  Then you can modify it however you want.


    If you say it can`t be done then i`ll try it

    • Marked as answer by Wilkyrl Wednesday, November 1, 2017 6:58 PM
    Wednesday, November 1, 2017 5:41 PM

All replies

  •  There is not a one size fits all solution to doing this.  How you do it or if it can be done at all depends on a few things like, what program it is,  is it actually the client area of the window you want to click on,  or is it a child control that is on the window?   If it is a child control,  what type of control is it?

    If you say it can`t be done then i`ll try it

    Friday, October 13, 2017 7:50 PM
  • OK, so I am using a windows form (the windows form project with visual studio) and want to click in the client. Does that help?
    • Edited by Wilkyrl Friday, October 13, 2017 7:51 PM
    Friday, October 13, 2017 7:51 PM
  • A minimized application is not in focus. So what do you mean by simulating a mouse click at a specific location in a minimized application?

    Is the minimized application an application you are writing and when it is minimized somehow you want a mouse click of either left, middle or right button to send a click to your minimized application in a certain window of said application? If so how can you denote that it is your app that should receive the mouse click and not some other app in focus in the main window of the PC?

    If that minimized app is not your app and you are writing an app that will be in focus and when in focus a specific mouse click will be sent to a certain window of a minimized 3rd party app that is different altogether.

    Please explain in some detail so as to keep the confusion to a minimum.

    Hola RAZERZ! It's almost Pecan harvest time....


    La vida loca

    Friday, October 13, 2017 8:03 PM
  • Sounds crazy?   :}

    What is it you want to accomplish? Is the minimized app your own or what is it? Why does not the user restore it and click it?

    PS Hi Monkey, Razerz !
    Friday, October 13, 2017 8:10 PM
  • Hola RAZERZ! It's almost Pecan harvest time....

    La vida loca

     Well..  look what the cat dragged in.  Time for some Pumpkin Pecan Pie.   8)

     Edit:  Hey Tom too.  8)


    If you say it can`t be done then i`ll try it

    • Edited by IronRazerz Friday, October 13, 2017 8:15 PM
    Friday, October 13, 2017 8:14 PM
  • Ok, to clarify:

    The minimized app is my own

    I do not want to minimize it, I want to inject a simulated click into the app, as to simulate a mouse click

    the app should not be in focus

    the app will only have 1 window 

    Hope this helps, let me know if you need more info

    Friday, October 13, 2017 8:22 PM
  • Ok, to clarify:

    The minimized app is my own

    I do not want to minimize it, I want to inject a simulated click into the app, as to simulate a mouse click

    the app should not be in focus

    the app will only have 1 window 

    Hope this helps, let me know if you need more info


     You did not give the full answer to the questions i asked in my last post.  I know you want to do it with a minimized window and it is one of your own apps but,  the other info i asked will be needed in order to give you an idea of the easiest/best way to try simulating the click.  Is it the actual form that you want to click on or is it a child control on the form you want to click on?  If it is a child control,  what type of control is it?

    If you say it can`t be done then i`ll try it

    • Edited by IronRazerz Friday, October 13, 2017 8:31 PM
    Friday, October 13, 2017 8:30 PM
  • My apologies (and thank you for the help so far).

    The form is what I want to click on, not any child control. See, I am using visual studio web browser, and so I have the form with the web browser, so I need to click on certain coordinates of said web browser which is built into the form. I think this is what you meant by form vs. control, let me know if it isn't.

    Friday, October 13, 2017 8:37 PM
  • Sounds crazy?   :}

    What is it you want to accomplish? Is the minimized app your own or what is it? Why does not the user restore it and click it?

    PS Hi Monkey, Razerz !

    Hola tommy!!! Weathers getting nice huh?

    La vida loca

    Friday, October 13, 2017 8:39 PM
  • Ok, to clarify:

    The minimized app is my own

    I do not want to minimize it, I want to inject a simulated click into the app, as to simulate a mouse click

    the app should not be in focus

    the app will only have 1 window 

    Hope this helps, let me know if you need more info

    Okay. And what if other windows are open on the desktop such as Notepad or Paint or an email window or whatever and one of them will be focused (at the top of the z order)? I realize you may not know how to write code for what you desire but you need to understand that there is more to this than meets your eye and without specific issues being presented that may need to be resolved in order to assist you accurately then the issue will become lengthy as you try and fail with provided code to what you say you need when in reality you may not know enough about apps and systems, etc, to describe exactly what is to occur. It is unlikely no other app will ever be in the main window of the desktop when you want to send a mouse click to a minimized app you wrote. And even if no other window is open on the desktop a left click, on an icon on the desktop or in the task bar can cause something to launch and a right click can bring up a desktop window too like below if no other apps are open on the desktop.


    La vida loca

    Friday, October 13, 2017 8:46 PM
  • My apologies (and thank you for the help so far).

    The form is what I want to click on, not any child control. See, I am using visual studio web browser, and so I have the form with the web browser, so I need to click on certain coordinates of said web browser which is built into the form. I think this is what you meant by form vs. control, let me know if it isn't.

     Just to make sure,  these are two different applications right?  One application that has the WebBrowser and another separate application that is simulating the click,  or is this all being done in the same application?

     Any controls you add onto the form are child controls of the form including a WebBrowser control.  I have simulated mouse clicks on a few different types of controls on minimized windows but,  never a WebBrowser control.  At this point,  i can only say that you might maybe be able to do it using UI Automation but,  i give no guarantee on that.

     

     EDIT:  If these are two separate apps,  you may be better off using a form of inter process communication to communicate between your two apps.  Then you can invoke a click on the html element of the document being shown in the WebBrowser.

     None of these methods are going to be easy to use/understand if you are not fairly well experienced.


    If you say it can`t be done then i`ll try it

    • Edited by IronRazerz Friday, October 13, 2017 9:12 PM
    Friday, October 13, 2017 9:03 PM
  • My apologies (and thank you for the help so far).

    The form is what I want to click on, not any child control. See, I am using visual studio web browser, and so I have the form with the web browser, so I need to click on certain coordinates of said web browser which is built into the form. I think this is what you meant by form vs. control, let me know if it isn't.

     Just to make sure,  these are two different applications right?  One application that has the WebBrowser and another separate application that is simulating the click,  or is this all being done in the same application?

     Any controls you add onto the form are child controls of the form including a WebBrowser control.  I have simulated mouse clicks on a few different types of controls on minimized windows but,  never a WebBrowser control.  At this point,  i can only say that you might maybe be able to do it using UI Automation but,  i give no guarantee on that.

     


    If you say it can`t be done then i`ll try it

    Oh, my bad. Simulate a mouse click... mouse_event (user32) ???

    Also see VB.Net forum search for mouse_event selecting Most recent thread


    La vida loca

    Friday, October 13, 2017 9:14 PM
  • Oh, my bad. Simulate a mouse click... mouse_event (user32) ???

    Also see VB.Net forum search for mouse_event selecting Most recent thread


    La vida loca

     The mouse_event and SendInput api functions might work if the app with the WebBrowser control is not minimized.  8)

    If you say it can`t be done then i`ll try it

    Friday, October 13, 2017 9:39 PM
  • thank you, I'll look into it
    Friday, October 13, 2017 9:56 PM
  • Hi Wilkyrl,

    Have you solved your issue now? if yes, please remember to close your thread by marking the helpful post as answer, it is beneficial to other community members who face the same issue.

    Thanks for your understanding.

    Best Regards,

    Cherry


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Thursday, October 19, 2017 8:14 AM
    Moderator
  • unfortunately, I have only gotten answers on how to click on a window with focus, not a window that is minimized or a background process.
    Thursday, October 19, 2017 1:21 PM
  • Ok, to clarify:

    The minimized app is my own

    I do not want to minimize it, I want to inject a simulated click into the app, as to simulate a mouse click

    the app should not be in focus

    the app will only have 1 window 

    If it has only 1 main window and it is minimized, it has no sense to simulate a click... on nothing.

    You can send messages to child windows, but if there are none...

    Thursday, October 19, 2017 1:47 PM
  • thank you, I'll look into it

    Well if you want to click on a document somewhere which is being displayed within a WebBrowser control then there is code that can perform that. However you would have to know what on the document is supposed to be clicked on such as an HTML documents (see W3 Schools) HTML Links which are "Hyperlinks" located on the document. Also the document may contain I suppose they would be called Hyperlinked images, videos, etc. But you would probably need to be very specific in describing exactly what you desire and possible provide a URL to the WebPage you intend to perform this with so others could view the source code to see if the possibility exists of performing what you want.

    I only come on here once a month really so the delay is my fault.

    And then rather than a mouse click to the HTML document displayed in the WebBrowser control you could possibly set up a Hot Key in some fashion to codewise "auto click" on the link in the WebPage displayed in the WebBrowser control.

    IronRazerz knows about that stuff but since you are unclear on the exact detail necessary for a programmer to assist you then everybody is guessing what you mean by "My apologies (and thank you for the help so far).

    The form is what I want to click on, not any child control. See, I am using visual studio web browser, and so I have the form with the web browser, so I need to click on certain coordinates of said web browser which is built into the form. I think this is what you meant by form vs. control, let me know if it isn't." as a WebBrowser control can be mouse clicked on in various locations as can a WebPage in a WebBrowser control if the WebPage has something clickable but since a WebPage is typically written in HMTL/CSS/Java Script I suppose then others could view the source code of the WebPage if provided a link to it to see which area you desire to initiate a link within the document.

    If none of that seems as what you desire then please explain further.


    La vida loca

    • Marked as answer by Wilkyrl Wednesday, November 1, 2017 7:00 PM
    Wednesday, November 1, 2017 4:40 PM
  • unfortunately, I have only gotten answers on how to click on a window with focus, not a window that is minimized or a background process.

     You have not really answered the questions i asked but,  i will take a wild guess that these really are two separate applications that you made and not just one application with two forms which would be much easier than communicating between separate applications.

     Below is an example of using the COPYDATASTRUCT structure,  the WM_COPYDATA message, and the SendMessage function for doing this using Interprocess Communications (as i mentioned earlier).  There are other methods for interprocess communication so,  this is just one way of doing it which communicates directly between your two applications.  Below is the example code for both applications,  the first one below is the application with the WebBrowser control on the form.

    Imports System.Runtime.InteropServices
    
    Public Class Form1
        Private Const WM_COPYDATA As Integer = &H4A
    
        <StructLayout(LayoutKind.Sequential)>
        Public Structure COPYDATASTRUCT
            Public dwData As IntPtr
            Public cbData As Integer
            Public lpData As IntPtr
        End Structure
    
        Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
            WebBrowser1.ScriptErrorsSuppressed = True
            WebBrowser1.Navigate("https://technet.microsoft.com/en-us/library/bb490996.aspx")
        End Sub
    
        Protected Overrides Sub WndProc(ByRef m As Message)
            MyBase.WndProc(m)
    
            If m.Msg = WM_COPYDATA Then 'check if the WM_COPYDATA message was sent to this application's window
    
                Dim cds As COPYDATASTRUCT = CType(m.GetLParam(GetType(COPYDATASTRUCT)), COPYDATASTRUCT) 'get the COPYDATASTRUCT that was passed to the lParam
    
                Dim Data(cds.cbData - 1) As Byte 'create a byte array of the size that the cbData was set to by the application that sent the message
                Marshal.Copy(cds.lpData, Data, 0, Data.Length) 'copy the byte array from the unmanaged memory pointed to by the lpData member
    
                Dim ElementInnerText As String = System.Text.Encoding.ASCII.GetString(Data) 'Convert the bytes back into a string. This is the text that will identify the html element to click.
    
                'get the html element from the document that is loaded in the WebBrowser and invoke a 'click' on it.
                Dim elm As HtmlElement = (From el As HtmlElement In WebBrowser1.Document.All.Cast(Of HtmlElement) Where el.InnerText = ElementInnerText).FirstOrDefault
                elm.InvokeMember("click")
            End If
        End Sub
    End Class
    
     

     This is the second application which has one button on it that will send the message to the WebBrowser application.

    Imports System.Runtime.InteropServices
    
    Public Class Form1
        Private Const WM_COPYDATA As Integer = &H4A
    
        <DllImport("user32.dll", EntryPoint:="SendMessageW")>
        Private Shared Function SendMessageW(ByVal hWnd As IntPtr, ByVal Msg As UInteger, ByVal wParam As IntPtr, ByRef lParam As COPYDATASTRUCT) As Integer
        End Function
    
        <StructLayout(LayoutKind.Sequential)>
        Private Structure COPYDATASTRUCT
            Public dwData As IntPtr
            Public cbData As Integer
            Public lpData As IntPtr
        End Structure
    
        Private cds As New COPYDATASTRUCT
    
        Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
            Dim p As Process = Process.GetProcessesByName("test click webbrowser app").FirstOrDefault 'get the first process of the application that has the WebBrowser on it
    
            If p IsNot Nothing Then 'make sure the process was found
    
                Dim bts() As Byte = System.Text.Encoding.ASCII.GetBytes("Wiki") 'get a byte array of a string that will identify the html element you want to click in the other app's WebBrowser control
    
                cds.cbData = bts.Count 'set this to the number of byte so that the other app know how many bytes to read
                cds.lpData = Marshal.AllocHGlobal(bts.Length) 'allocate enough unmanaged memory to hold the byte array
                Marshal.Copy(bts, 0, cds.lpData, bts.Length) 'copy the byte array to the allocated memory pointed to by the lpData member of the COPYDATASTRUCT
    
                SendMessageW(p.MainWindowHandle, WM_COPYDATA, Me.Handle, cds) 'send the WM_COPYDATA message to the other application's handle, passing the COPYDATASTRUCT in the lParam
    
                Marshal.FreeHGlobal(cds.lpData) 'now free the unmanaged memory
            Else
                MessageBox.Show("The other application was not found.")
            End If
        End Sub
    End Class
    
     

     Here you can see it working....

     

     You will need to stud the code,  research what you don't understand,  and experiment a little with it until you understand what it does and how it does it.  Then you can modify it however you want.


    If you say it can`t be done then i`ll try it

    • Marked as answer by Wilkyrl Wednesday, November 1, 2017 6:58 PM
    Wednesday, November 1, 2017 5:41 PM