none
How do I interpret touch screen HID data RRS feed

  • Question

  • Hi,

    I am trying to track touch screen interaction in my program (the device is Surface Pro 3).

    I have been able to successfully register my device for Raw HID input notification and get back WM_INPUT messages when the screen is touched. However I do not know how to interpret this data and get touch screen coordinates.

    What i get is a byte array (seems to be 27 bytes long).

    As you can see from another post of mine I have had a little bit of success however I am not sure if am on the right path.

    For each touch input I get tagRAWHID.dwSizeHid = 27 and tagRAWHID.dwCount = 1;

    I assumed that bytes 6 to 10 were the x coordinate and 10 to 14 were the y coordinate. I feel this might be right becaue if I touch the left edge of the screen and drag till the very right edge, the x coordinate goes from 0 to 629155200 and vice versa.

    My remaining problem is how do I detect when the right edge has been touched. When i ran the code on my Surface Pro 3, the right edge x-coordinate was 629155200 but running the same code on 7" windows 10 Tab the right edge is 9.


    This is what the code looks like. Feel free to review it on github, its a code project sample that i modified to test out the HID touch input

    rawtouch.cs

     public void ProcessRawInput(IntPtr hdevice)
            {
                var size = 0;

                // Determine Size to be allocated

                var ret = Win32.GetRawInputData(hdevice, DataCommand.RID_INPUT, IntPtr.Zero,  ref size, Marshal.SizeOf(typeof(Rawinputheader)));

                if (ret == -1)
                {
                    Console.WriteLine("error");
                    return;
                }

                int sizeToAllocate = Math.Max(size, Marshal.SizeOf(typeof(RawInput_Marshalling)));

                IntPtr pData = Marshal.AllocHGlobal(sizeToAllocate);
                try
                {
                    //Populate alocated memory
                    ret = Win32.GetRawInputData(hdevice, DataCommand.RID_INPUT, pData, ref sizeToAllocate, Marshal.SizeOf(typeof(Rawinputheader)));

                    if (ret == -1)
                    {
                        throw new System.ComponentModel.Win32Exception();
                    }

                    Rawinputheader header = (Rawinputheader) Marshal.PtrToStructure(pData, typeof(Rawinputheader));

                    //RAWINPUT starts with RAWINPUTHEADER, so we can do this
                    RawInputDeviceType type = (RawInputDeviceType)header.dwType;
                    switch (type)
                    {
                        case RawInputDeviceType.RIM_TYPEHID:
                            {
                                //As described on page of RAWHID, RAWHID needs special treatement
                                RawInput_Marshalling raw = (RawInput_Marshalling) Marshal.PtrToStructure(pData, typeof(RawInput_Marshalling));

                                //Get marshalling version, it contains information about block size and count
                                RawInput_NonMarshalling raw2 = default(RawInput_NonMarshalling);

                                //Do some copying
                                raw2.header = raw.header;
                                raw2.hid.dwCount = raw.hid.dwCount;

                                raw2.hid.dwSizHid = raw.hid.dwSizHid;

                                var numBytes =  raw.hid.dwCount * raw.hid.dwSizHid;
                                byte[] data = new byte[numBytes];

                                //Allocate array
                                //Populate the array
                                IntPtr rawData =  new IntPtr(
                                    pData.ToInt64() + Marshal.SizeOf(typeof(Rawinputheader)) + Marshal.SizeOf(typeof(Rawhid_Marshalling)));

                                Marshal.Copy(rawData, data, 0, (int)numBytes);

                                // Extract X & Y
                                byte[] xBytes = new byte[4];
                                Buffer.BlockCopy(data, 6, xBytes, 0, 4);
                                byte[] yBytes = new byte[4];
                                Buffer.BlockCopy(data, 10, yBytes, 0, 4);


                                int x = BitConverter.ToInt32(xBytes, 0);
                                int y = BitConverter.ToInt32(yBytes, 0);


                                Console.WriteLine($"X: {x}\tY: {y}");
                                if (TouchActivated != null)
                                {
                                    TouchActivated(this, new RawInputEventArg(x, y));
                                }
                                //return raw2;
                                break;
                            }
                        default:
                            {
                                //No additional processing is needed
                                var x = (RawInput_Marshalling)Marshal.PtrToStructure(pData, typeof(RawInput_Marshalling));
                                break;
                            }
                    }
                }
                finally
                {
                    Marshal.FreeHGlobal(pData);
                }
            }

                
    Wednesday, February 3, 2016 4:33 PM

Answers

  • just read the HID section of the WDK. some of the HID examples may be helpful as well. I don't know if you will be able to read global touch gestures, the OS may consume them exclusively

    d -- This posting is provided "AS IS" with no warranties, and confers no rights.

    Thursday, February 4, 2016 5:42 PM
  • you need to use the HID.dll APIs to parse the buffer. you are probably better off using WM_TOUCH which is baked//processed touch data

    d -- This posting is provided "AS IS" with no warranties, and confers no rights.

    Wednesday, February 3, 2016 5:24 PM

All replies

  • you need to use the HID.dll APIs to parse the buffer. you are probably better off using WM_TOUCH which is baked//processed touch data

    d -- This posting is provided "AS IS" with no warranties, and confers no rights.

    Wednesday, February 3, 2016 5:24 PM
  • Thanks for the reply Doron

    From what I have read about WMTouch, I wont be able to detect touch guestures when my application is minimized. What I want to do is to recognize a gesture on the OS and then bring my application to the foreground (like the Windows Action Center).

    I will take a look at HID.dll, do you know of any good articles for me to start of with?

    Tariq

    Thursday, February 4, 2016 7:04 AM
  • just read the HID section of the WDK. some of the HID examples may be helpful as well. I don't know if you will be able to read global touch gestures, the OS may consume them exclusively

    d -- This posting is provided "AS IS" with no warranties, and confers no rights.

    Thursday, February 4, 2016 5:42 PM
  • Hi,

    I am trying to track touch screen interaction in my program (the device is Surface Pro 3).

    I have been able to successfully register my device for Raw HID input notification and get back WM_INPUT messages when the screen is touched. However I do not know how to interpret this data and get touch screen coordinates.

    What i get is a byte array (seems to be 27 bytes long).

    As you can see from another post of mine I have had a little bit of success however I am not sure if am on the right path.

    For each touch input I get tagRAWHID.dwSizeHid = 27 and tagRAWHID.dwCount = 1;

    I assumed that bytes 6 to 10 were the x coordinate and 10 to 14 were the y coordinate. I feel this might be right becaue if I touch the left edge of the screen and drag till the very right edge, the x coordinate goes from 0 to 629155200 and vice versa.

    My remaining problem is how do I detect when the right edge has been touched. When i ran the code on my Surface Pro 3, the right edge x-coordinate was 629155200 but running the same code on 7" windows 10 Tab the right edge is 9.


    This is what the code looks like. Feel free to review it on github, its a code project sample that i modified to test out the HID touch input

    rawtouch.cs

     public void ProcessRawInput(IntPtr hdevice)
            {
                var size = 0;

                // Determine Size to be allocated

                var ret = Win32.GetRawInputData(hdevice, DataCommand.RID_INPUT, IntPtr.Zero,  ref size, Marshal.SizeOf(typeof(Rawinputheader)));

                if (ret == -1)
                {
                    Console.WriteLine("error");
                    return;
                }

                int sizeToAllocate = Math.Max(size, Marshal.SizeOf(typeof(RawInput_Marshalling)));

                IntPtr pData = Marshal.AllocHGlobal(sizeToAllocate);
                try
                {
                    //Populate alocated memory
                    ret = Win32.GetRawInputData(hdevice, DataCommand.RID_INPUT, pData, ref sizeToAllocate, Marshal.SizeOf(typeof(Rawinputheader)));

                    if (ret == -1)
                    {
                        throw new System.ComponentModel.Win32Exception();
                    }

                    Rawinputheader header = (Rawinputheader) Marshal.PtrToStructure(pData, typeof(Rawinputheader));

                    //RAWINPUT starts with RAWINPUTHEADER, so we can do this
                    RawInputDeviceType type = (RawInputDeviceType)header.dwType;
                    switch (type)
                    {
                        case RawInputDeviceType.RIM_TYPEHID:
                            {
                                //As described on page of RAWHID, RAWHID needs special treatement
                                RawInput_Marshalling raw = (RawInput_Marshalling) Marshal.PtrToStructure(pData, typeof(RawInput_Marshalling));

                                //Get marshalling version, it contains information about block size and count
                                RawInput_NonMarshalling raw2 = default(RawInput_NonMarshalling);

                                //Do some copying
                                raw2.header = raw.header;
                                raw2.hid.dwCount = raw.hid.dwCount;

                                raw2.hid.dwSizHid = raw.hid.dwSizHid;

                                var numBytes =  raw.hid.dwCount * raw.hid.dwSizHid;
                                byte[] data = new byte[numBytes];

                                //Allocate array
                                //Populate the array
                                IntPtr rawData =  new IntPtr(
                                    pData.ToInt64() + Marshal.SizeOf(typeof(Rawinputheader)) + Marshal.SizeOf(typeof(Rawhid_Marshalling)));

                                Marshal.Copy(rawData, data, 0, (int)numBytes);

                                // Extract X & Y
                                byte[] xBytes = new byte[4];
                                Buffer.BlockCopy(data, 6, xBytes, 0, 4);
                                byte[] yBytes = new byte[4];
                                Buffer.BlockCopy(data, 10, yBytes, 0, 4);


                                int x = BitConverter.ToInt32(xBytes, 0);
                                int y = BitConverter.ToInt32(yBytes, 0);


                                Console.WriteLine($"X: {x}\tY: {y}");
                                if (TouchActivated != null)
                                {
                                    TouchActivated(this, new RawInputEventArg(x, y));
                                }
                                //return raw2;
                                break;
                            }
                        default:
                            {
                                //No additional processing is needed
                                var x = (RawInput_Marshalling)Marshal.PtrToStructure(pData, typeof(RawInput_Marshalling));
                                break;
                            }
                    }
                }
                finally
                {
                    Marshal.FreeHGlobal(pData);
                }
            }

                

    Hi, I am looking for a way to capture all touch on my screen globally.

    Until now, I found no solution that works globally, does your solution works globally?

    Saturday, March 31, 2018 7:39 AM
  • What bigger problem are you trying to solve?

    d -- This posting is provided "AS IS" with no warranties, and confers no rights.

    Sunday, April 1, 2018 5:15 AM
  • I have the same problem and I don't know how to get touch point coordinates. I've managed to get preparsed data with GetRawInputDeviceInfo(raw->header.hDevice, RIDI_PREPARSEDDATA, preparsedData, &data_size);, but nothing useful from this.

    Have you solved this? Thanks.

    Tuesday, February 26, 2019 7:25 PM
  • I want to log touches from my monitor to find out if my monitor is defective. It sends me multiple point at once when using JavaFX, so I'm writing this "touch listener" in cpp to find out, if monitor or Java is the problem.

    Tuesday, February 26, 2019 7:27 PM