none
Implementing Remote Desktop functionality (not RDP) RRS feed

  • Question

  • I need to look into implementing a remote desktop functionality for physical PCs and virtual PCs. The goal is to integrate (virtual) Windows PCs into an existing "KVM over IP" system.

    Until today physical PCs are integrated by using a hardware device connected to the video port of a PC. The video signal is then trancoded by this device to a IP-based protocol, which is sent to a receiver, that converts the IP-based datastream back to a video signal. The transmitter and receiver devices also handle keyboard and mouse data in a similar way. 

    While this approach works very well with physical PCs (and is completely OS independant), it cannot be used with virtual PCs, because they don't have a physical video port. Therefore I need to implement some software parts, that handles video, keyboard and mouse information and translate them into the used proprietary protocol. 

    Certainly some members of this forum already implemented similar software and could give me some guidance what to start with. 

    The Windows video driver model changed significantly with Windows 8. We don't need to consider Windows 7, but can concentrate on the Windows 8.1 and Windows 10 versions (WDDM). I read already some articles and it seems that there are several approches/technologies to implement a remote display functionality, like:

    • Desktop Duplication API
    • Remote Display Driver
    • WDDM Display Only Driver
    • Virtual WDDM Adapter Driver
    • WDDM Filter Hook Driver

    It seems that the Desktop Duplication API is the way to go, but I'm not completely convinced, especially when used in virtual machines. Will this work on virtual machines running on a hyperviser (e.g. Hyper-V)?

    What drivers, APIs should I look into for the keyboard and mouse data part of this remote desktop implementation?

    Any help, ideas, experiances are very much appreciated.

    Thank you in advance

    Markus

    Wednesday, January 30, 2019 10:36 AM

Answers

  • Yes, the Desktop Duplication APIs are the best way to implement this. As I recall, there are a few implementations of this on GitHub

     -Brian


    Azius Developer Training www.azius.com Windows device driver, internals, security, & forensics training and consulting. Blog at www.azius.com/blog

    Wednesday, January 30, 2019 6:39 PM
    Moderator

All replies

  • Yes, the Desktop Duplication APIs are the best way to implement this. As I recall, there are a few implementations of this on GitHub

     -Brian


    Azius Developer Training www.azius.com Windows device driver, internals, security, & forensics training and consulting. Blog at www.azius.com/blog

    Wednesday, January 30, 2019 6:39 PM
    Moderator
  • Hi Brian 

    Thank you very much for your answer. So the Desktop Duplication API is the way to go! Seems that you know this field quite well (saw some of your other posts). Do you know whether keyboard and mouse is also handled by the Desktop Duplication API? If not what APIs should be used for bringing mouse and keyboard data to the remote computer.?

    I already bookmarked some links on Github and started to look into it. 

    Markus

    Friday, February 1, 2019 10:13 AM
  • The DD APIs only provide changes in the display surface. Keyboard and mouse are up to you.

    How will your implementation be superior to the built-in (and free) Remote Desktop, which already does everything you mentioned, and more?

     -Brian


    Azius Developer Training www.azius.com Windows device driver, internals, security, & forensics training and consulting. Blog at www.azius.com/blog

    Friday, February 1, 2019 6:46 PM
    Moderator
  • Hi Brian

    Thank you for your answer concerning mouse and keyboard. I really appreciate it. 

    It will not be superior to the built-in Remote Desktop! It should implement a different protocol to transfer video, mouse, keyboard (and later audio and USB) over the LAN. The protocol is already used in an existing video distribution system and it's optimized for low latency, while RDP is rather optimized for low bandwidth. Until now propriety transmitter devices are used to convert video signals into this LAN protocol. In virtual PCs you don't have physical video signals and the idea is to convert the "desktop" into this LAN protocol by software. 

    Hope that my description is good enough to get the point across. Actually I think it would be presumptuous to suppose that I could just come and implement something that is superior to the built-in Remote Destop feature of Windows. 

    Markus

    Saturday, February 2, 2019 8:04 PM
  • "How will your implementation be superior to the built-in (and free) Remote Desktop, which already does everything you mentioned, and more?"

    It's not free for Home editions.  You have to have upgrade a Pro or Enterprise version to get it.


    Friday, March 29, 2019 11:24 PM