none
Using one test machine for multiple OSes with HCK RRS feed

  • Question

  • I work for one of the major USB microcontroller vendors, and am trying to do some USB driver package certification testing with the HCK.

    A few years ago, I did similar testing with the WLK, and I was able to use two computers to test all OSes (one machine running Windows Server 2003, and with the test machine equipped with a removable hard drive rack, with each hard drive containing one OS installation each).

    I have recently observed and read that the HCK does not support this scenario, and instead requires that multiple unique test machines are used (one for each OS varient).  Since I want to test on Win 7 and 8 (32-bit and 64-bit), it appears I need four unique test machines (in addition to the Win 2k8R2 server machine), and that all the test machines must be modern with USB 3.0 host controllers.

    Unfortunately, the semiconductor economy is doing very poorly right now, and many companies are laying off workers and have instituted spending freezes, my company included.  Although I have managed to convince management that it would be a good idea to WHQL some USB drivers, there is no way I will get budget approval to buy four new test machines to do all of the testing (although I already have approval for the Microsoft submission fee).

    Therefore, my question is, can I work around the HCK limitation, and do all of the necessary OS testing on a single test machine? 

    My personal computer at home is modern enough to have a USB 3.0 host controller and can pass the tests, but I only have access to this one test machine (along with a Win 2008 R2 server machine).  Can I run all the tests on one OS varient, get them all passing, save the package, then rinse and repeat for each OS variant (Win7/8 32/64bit)?  Once I have all four separate packages, can I use the "merge package" feature in the HCK Studio to bundle up a single submission package?  If I do this, will the package be accepted by Microsoft, or will they take my test submission fee and reject the package since all tests for all OSes were conducted on a single PC?

    Any advice would be appreciated.  I've already invested considerable company time and money (ex: on an MSDN OS subscription) and don't want it all to have been a waste.

    Tuesday, December 4, 2012 12:45 AM

Answers

  •     I do this as well.  You have the ability to run multiple operating systems on the same system I have been using a method of booting to a Safe first partition then installing a new Operating system on the second partition via scripts all via the DTM/HCK.  Once the new OS is installed the DTM/HCK gets silently installed all via the Unattended file.  Most of the time this works flawlessly. Only once in a while to fix DTM/HCK connectivity I put the system in "Unsafe mode" in the DTM/HCK and reboot the computer, the DTM/HCK then reconnects and sees the new Operating system and goes to "Manual mode".  I then place the system in reset mode and wait for it to go ready, and it's ready to run.  Worst case scenario if that does not work.  Unsafe mode uninstall the DTM/HCK on client and reinstall it, DTM/HCK sets it to Manual mode and you reset it to ready.  The key when you have issues is always unsafe and reboot or reinstall to reinitialize the connection from client to server. 

     

         Works well have been doing this since DTM 1.2 and still works on HCK 2.0.  I use Unattended installs over having installed on many partitions as its always a fresh clean install to run my tests.

     

    • Proposed as answer by JB-Matrox Wednesday, December 5, 2012 4:06 PM
    • Marked as answer by Asdf123man Wednesday, December 5, 2012 5:52 PM
    Wednesday, December 5, 2012 4:06 PM

All replies

  • Of course, you can use the same test platform for device test if it can works fine on all OS.

    Actually, I use the same test platform for WinXP 32/64, Vista 32/64, Win7 32/64, Win8 32/64 device HCK test. It's no any problem.

    BTW, you can add-on PCI-E USB 3.0 card for USB device test.

    Thanks, Steve

    Tuesday, December 4, 2012 1:50 AM
  •     I do this as well.  You have the ability to run multiple operating systems on the same system I have been using a method of booting to a Safe first partition then installing a new Operating system on the second partition via scripts all via the DTM/HCK.  Once the new OS is installed the DTM/HCK gets silently installed all via the Unattended file.  Most of the time this works flawlessly. Only once in a while to fix DTM/HCK connectivity I put the system in "Unsafe mode" in the DTM/HCK and reboot the computer, the DTM/HCK then reconnects and sees the new Operating system and goes to "Manual mode".  I then place the system in reset mode and wait for it to go ready, and it's ready to run.  Worst case scenario if that does not work.  Unsafe mode uninstall the DTM/HCK on client and reinstall it, DTM/HCK sets it to Manual mode and you reset it to ready.  The key when you have issues is always unsafe and reboot or reinstall to reinitialize the connection from client to server. 

     

         Works well have been doing this since DTM 1.2 and still works on HCK 2.0.  I use Unattended installs over having installed on many partitions as its always a fresh clean install to run my tests.

     

    • Proposed as answer by JB-Matrox Wednesday, December 5, 2012 4:06 PM
    • Marked as answer by Asdf123man Wednesday, December 5, 2012 5:52 PM
    Wednesday, December 5, 2012 4:06 PM
  • I also use the same machine(s) (1 client and 1 support) for multiple OS testing. My method however is to finish an OS as its own project, and sign/save the results. Then run the other OS and save those results. You can then use the HCK studio to 'merge' packages, and sign the final package for submission to the sysdev dashboard. This has not caused any issues for my submissions yet.
    Wednesday, December 5, 2012 7:03 PM
  • How much time does it take to pass all the tests on all the OS's on the same (Client) machine? Some WHCK 2.0 tests have really long runtime like Oplocks test, Sleep and PnP test, etc. If the runtime of all the tests on all the OS's on single machine is fine then I also would like to follow this method.

    Thanks,

    Nilesh

    Thursday, December 6, 2012 8:07 AM
  • The time it takes to run the tests will of course depend on what tests are required for your product.  However, in my case, for a simple WinUSB device with no special functionality, the HCK estimates it will take about 42 hours to complete all tests (for a single OS variant).  In practice, if you know exactly what you are doing, have already fixed all issued with INF files, etc., and you have a completely rock solid setup (no hangs or anything during the tests), it appears to take around 9 hours to complete all of the tests requried for my device.

    Of course, in the real world problems happen, systems aren't always perfect, and when you run into problems huge amounts of time can be wasted.  Ironically, I think the HCK estimated 42 hours can end up being fairly accurate in many cases, due to the wasted time when you hit a snag.

    Friday, December 7, 2012 3:59 AM
  • ofcourse, you can use same hardware to run multiple OSes certification. Run one OS first, complete it and save the logs. Boot the same hardware with different OS, create new project , run the test and save the logs again.

    Thanks,

    Mudit

    Tuesday, November 5, 2013 9:29 AM