none
How to deploy a Windows Kernel Mode Driver to a VM? RRS feed

  • Question

  • Dear all,

    first of all, I am new to Windows Kernel Mode Driver development. So, I just wanted to start with a downloaded sample driver from the MSDN (inspect - Windows Filtering Platform). I have set up my environment as described in the following MSDN description [ http://msdn.microsoft.com/en-us/library/windows/hardware/ff538143%28v=vs.85%29.aspx ]. So my setup looks as follows:

    Win 7 (64bit) [Host]     --- named pipe -->      Win 7 (32bit) [VM]

    Furthermore, in the VM I did the following tasks:

    • disabled UAC by setting the corresponding registry key to 0
    • enabled administrator account and set a password

    On the host OS I have VS2012 from which I want to deploy the built kernel mode driver to the the target VM. I followed the instructions how to configure VS2012 for deploying kernel mode drivers [ http://msdn.microsoft.com/en-us/library/windows/hardware/hh454835%28v=vs.85%29.aspx ]

    In VS2012, I can build the driver but unfortunately deploying fails:

    1>  Deploying driver files for project "D:\WFP\Examples\TrafInsp\C++\inspect Package\inspect Package.vcxproj".  Deployment may take a few minutes...
    1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V110\Platforms\Win32\ImportAfter\DriverDeployment8.0.targets(69,9): error : Could not connect to the remote computer for deployment.
    1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V110\Platforms\Win32\ImportAfter\DriverDeployment8.0.targets(69,9): error : Object reference not set to an instance of an object.

    Any hints are appreciated! I am struggling with this error for a few days...

    Thanks in advance,

    Sven




    Tuesday, June 25, 2013 2:03 PM

All replies

  • Hello Sven,

    Hope it will help you.

    Kernel Mode and User Mode driver debugging using Visual Studio 2012 (VS 2012) and target Virtual Machine (VM) onto  the same Host Machine

     

    Scenario:

    Host:       Windows 7 Ultimate x64, (Please confirm the settings if selecting Win7 Pro or Win7 Home)

    Target:    Windows 7 Ultimate x64, Virtual Machine on the same Host by using VMware Workstation 9

     

    Requirement:

    Windows 7 Ultimate x64 or x86

    VMware Workstation 9

    .NET Framework 4.5

    Visual Studio 2012 Ultimate, (again, please confirm the settings if selecting Pro or Home)

    WDK 8

    WDK 8 Redistributable Components

     

    Installation and creation of proper environment is your part, so please take care. But still, if require any help can ask.

     

    Host Machine Preparation:

    Install all these tools in the sequence as given below on the Host Machine.

    .NET Framework 4.5

    Visual Studio 2012

    WDK 8

    WDK 8 Redistributable Components

     

    Target Machine Preparation:

    Install

    VMware Workstation 9, on the same Host Machine

    Create new Virtual Machine

     

    Kindly refer proper respective website for in detail depth knowledge.

     

    After installation and creation of Virtual Machine, we will go through the setup of VM and VS 2012 debug environment.

     

    Setting up Virtual Machine:

    Here I’m using virtual serial port communication’s setting to debug your Kernel Mode drivers on Virtual Machine. For this my special thanks to Mr. Michael Engstler, you can found his in detail blog here

     

    Shut down your VM, and then go to

    VM > Settings

    On Virtual Machine Setting Page

    Hardware > Add > Serial Port > Output to named pipe

    Named pipe = \\.\pipe\com_2

    First Combo box = The end is the server

    Second Combo box = The other end is an application

    Connect at power on = Checked

    Finish

    Yield CPU on poll = Checked

     

    After adding Serial Port,

     

    In VM, in an elevated Command Prompt window, enter the following commands:

    bcdedit /debug on

    bcdedit /dbgsettings serial debugport:2 baudrate:115200

     

    In VM, search for "Manage advanced sharing settings":

    Set "Turn on file and printer sharing"

    Set "Turn off password protected sharing"

    In VM, search for "Computer Management":

    Local Users and Groups -> Users:

                    Administrator -> Right Click -> Properties -> Uncheck Account is disabled

     

                    Administrator -> Right Click -> Set Password -> Set your password

    To take these changes into effect reboot VM.

     

    Setting up Visual Studio

    Now come to Host, and start writing on your first Kernel Mode driver as given on msdn and follow the same steps very carefully.

    After build of your “KmdfSmall” Kernel Mode driver; check the below for configuring your Visual Studio for debugging and then rest of the steps are same

     

    Configure Visual Studio

    Onto your project “KmdfSmall”, go to

    Driver-> Test -> Configure Computers -> Add New Computer:

    Computer name = VM name

    Select Provision computer and choose debugger settings(2nd option)

    Next

    Connection Type = Serial

    Pipe = Checked

    Reconnect = Checked

    Port = \\.\pipe\com_2

    Target Port = com2

    Next -> Finish

     

    It will take some time and may prompt you for the first time to Username and Password, then simply enter “Administrator” & Password as you entered in above step while setting up Virtual Machine.

     

    Driver Deployment

    Now, after configuration follow the same steps as given on msdn for driver deployment. Don’t press break all and F5 here. Till your Visual Studio is into debug mode go to VM.

     

    Test driver on Virtual Machine

    Now on VM, you will find VM has been logged in as “WDKRemoteUser” and in C:\ drive a folder named as C:\DriverTest\Drivers

    has been created.

     

    This folder should contain all the four files KmdfSmall.cat, KmdfSmall.sys, KmdfSmall.inf and WdfCoinstaller01011.dll.

     

    Now go to device manager and check your “KmdfSmall” driver entry is there into Samples heading. If device manager contain “KmdfSmall” driver entry into Samples heading disable the driver.

     

    It will prompt for reboot the machine just reboot your VM meanwhile your Visual Studio would be into debug mode onto Host. After rebooting VM go to Host and in Visual Studio press break all and F5 now.

     

    Still the debug won’t begin at this time. So go to VM and enable your driver. As soon as you enable your driver it will jump or through the control to the Visual Studio on Host, but for this you have to put some break points in your code.  Thus control will hit the break point in your code.


    Regards

    Matt

     


    • Proposed as answer by Beginner15 Wednesday, June 26, 2013 3:25 AM
    Wednesday, June 26, 2013 3:24 AM
  • Hi Matt,

    first of all: thank you for your very detailed step-by-step description how to setup the host and target environment. Does the Win 7 OS version matter? My host is Win 7 Enterprise (64bit), whereas the target VM is running Win 7 Ultimate (32bit). Furthermore, I am using Visual Studio 2012 Professional on the host and an up-to-date VM player, not the VM workstation. I don't know if this also matters... In VM player it is also possible adding a virtual serial port mapped to a pipe (same configuration as you described).

    Best regards,

    Sven


    • Edited by svkers Wednesday, June 26, 2013 7:34 AM missing details
    Wednesday, June 26, 2013 6:53 AM
  • Hi Matt,

    I am one step further. The problem was the following: The user I was logged in at the host OS was a domain user which was not existent at the VM OS. Thus, I created a local user with the same credentials (username/password) as the user in the VM. Now, I get the following issue during provisioning the VM:

    RemoteExecute: Binary: %SystemDrive%\DriverTest\Setup\x86\vcredist_x86.exe
    RemoteExecution: Arguments: /q
    System.IO.IOException: Der Netzwerkpfad wurde nicht gefunden.

       at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
       at System.IO.Pipes.NamedPipeClientStream.Connect(Int32 timeout)
       at System.IO.Pipes.NamedPipeClientStream.Connect()
       at Microsoft.DriverKit.DriverTestServices.RemoteExecute(String computerName, RemoteExecuteCommand& remoteCommand)
       at Microsoft.DriverKit.DriverTestResult.ResultRun()
    Result status updated: Fail

    Should I install the vcredist-package by hand in the VM?

    Thanks,

    Sven

    Wednesday, June 26, 2013 8:19 AM
  • Before proviosioning, tell me one thing are you able to talk to your VM

    means from your host just use "ping VMname" command from command prompt

    regards

    matt

    • Proposed as answer by Beginner15 Thursday, June 27, 2013 5:33 AM
    Wednesday, June 26, 2013 10:36 AM