none
WHCK running on Multiple machines RRS feed

  • Question

  • HI ,

    Can I run the WHCK tests on more than one machine in a network at the same time.How much extra load it will have on server(How much extra time will it take to get all the tests done)

    Tx and Rgds,

    Rp

    Thursday, June 20, 2013 8:50 AM

All replies

  • Hi,

    Yes (If Distributable..!)

    The idea of running all the tests across different client machines is known as distributable testing. Many of the tests in the system are marked as being distributable. Unfortunately, this is not supported via HCK Studio. From the HCK FAQ page (http://msdn.microsoft.com/en-us/library/windows/hardware/hh852377.aspx), this is addressed by indicating that you can use the HCK Object Model.

    Q: After the project starts and all tests are scheduled, the HCK Studio does not allow for the addition of more machine (targeted devices). Adding machines doubles or triples the amount of tests needed to complete the project, and does not distribute the already scheduled tests to them.

    One solution to this distributed testing, which can be done using the object model. See the HCK API Reference.


    Yesubabu Gude

    • Proposed as answer by Yesubabu Gude Friday, June 21, 2013 7:58 AM
    Thursday, June 20, 2013 10:12 AM
  • There is no problem about running tests on several machines. As long as they are in the same network, in the same workgroup and connected to the server. The server charge is I believe pretty low, I don't think you'll experience any slows during tests.
    Friday, June 21, 2013 11:52 AM
  • There is no problem about running tests on several machines. As long as they are in the same network, in the same workgroup and connected to the server. The server charge is I believe pretty low, I don't think you'll experience any slows during tests.

    Hi,

    Did you tried, running tests on several machines as the same network that is connected to one server?

     Case-1: Take two machines that is connected one server then whenever you want run the tests it will run tests which is selected in selection tab of the device/system corresponding machine only if its  not distributable(ie at a time select only one machine only).

    case-2: If its distributable then run the both machines that connected to server.

    Example: NDISTest 6.5 -[2Machine]-ShortPackets and so on..


    Yesubabu Gude

    Friday, June 21, 2013 6:43 PM
  • I Think if some of us who has "Actually" done the test can answer this post correctly ! I havent got the two Machines tested at a single time .I will do it and update the figures - "time taken" and "the possibility" of test.

    Rgds.

    RP

    Monday, June 24, 2013 4:23 AM
  • Hi Rblooper,

    I tried and observed the behavior of this context and any way you will try and share the status.


    Yesubabu Gude

    Monday, June 24, 2013 4:35 AM
  • I don't understand what is you problem exactly ... 

    You want to know if you can run HCK tests on several machines, right ? Which means you want to connect several machines to your HCK Server and run tests on them, right ?

    Monday, June 24, 2013 8:14 AM
  • Hi Ingenico,

    That is we know dude..

    Here question is can we run the WHCK tests on more than one machine that connected to the server in a network at the same time..?


    Yesubabu Gude

    Monday, June 24, 2013 8:22 AM
  • Yes.

    In your machine pool you can have several machines. After selecting the devices you want to test related to your driver or watever you want to test, you can run all the different generated tests.

    Those tests will be ran on several machines, at the same time.

    Monday, June 24, 2013 8:31 AM
  • No...No..

    If its Distributable then it possible to run the more than one client machine that is connected to one server.

    If its not Distributable then we can't run...

    HCK Studio in selection tab after selecting the machine pool, we have to select the device that is connected to client machine right ( In this case if you select the Client1) then only we can able to run that Client1 only.. or if you select the client2 then able to run on client2 only....... otherwise if you select both tests then you will get repeated tests on Tests tab here also we can able to run the tests only at a time on selected clients. ( because I tried and come up with these results.)

    Better to take one scenario and try it and share the possible answers ( Did you tried your previous time???).


    Yesubabu Gude

    Monday, June 24, 2013 9:08 AM
  • When you select devices to test you get a generated list of tests.

    Those tests are related to on related test Client but you can run different tests on several clients at the same time. Likewise, you can run idendical tests on several clients at the same time. (i.e. you can run USB-IF test on one client and USB Topology Compatibility test on an other as well as USB-IF on two different clients).

    Since tests are only related to one client you can run several tests with the same name at the same time. If you have the same test for several clients, i.e. USB-IF for Win7x32 + Win7x64 + Win8x32 + Win8x64, you will have as much iteration as client to run the test on.

    Monday, June 24, 2013 9:26 AM
  • Hi Ingenico,

    I am still disagree with you.. anyway for the sake my clarity better to go question to answer like..

    1) while running tests on two clients that is connected one server before that in selection tab select the device which you want test that is connected to client right (Q:- Here which client( out of the client1 and client2) you are selecting..?)

    2) If you select client1 or client2 then corresponding client we can run the tests on the selected client only... (Q:- Am a right ??)

    3) If you select the both you will get repeated tests in tests tab(if both clients have same device ie device selected in selection tab of HCK Studio) ... I think this is default by design for the repeated tests.

    May be in this case we can run tests on both clients (client1 and client2) and this is not the recommended way.


    Yesubabu Gude

    Monday, June 24, 2013 10:30 AM
  • If you want to certify your product for several OS (i.e. Win8x32 Win8x64 Win7x32 Win7x64) you will have several tests with the same name in the test tab but NOT WITH the same target (again, Win8x32 Win8x64 Win7x32 Win7x64). Those test will have the same name but not the same ID (to avoid conflicts during the results gathering). 


    Monday, June 24, 2013 10:59 AM
  • Hi ,

    I am expecting the answers for the previous post... and present I am not bother about the several OS what you mentioned..anyway thanks.


    Yesubabu Gude

    Monday, June 24, 2013 11:04 AM
  • HI,

    Its a good debate here.Sorry i was not able to produce the results so far.

    I am summarizing again what I have asked and what I have inferred from your comments :

    My Question :

    Running same set of WHCK test cases (say for USB) on 4 machines ( one x86 windows 8 PC, one x64 windows 8 Tablet, one x64 windows 8 Phone and one more x64 windows 8 PC) at the same time

    Note : All devices are running windows 8 as you can see.

    ingenico_protocol : Yes

    Yesubabu Gude : Yes if it is Distributable .

    Now I am trying to figure it out how to make the system Distributable  !!

    Thanks for your comments and suggestions !!

    Thanks and Rgds,

    Rp


    • Edited by hcktester Wednesday, June 26, 2013 4:16 AM spell mistake
    Wednesday, June 26, 2013 4:15 AM
  • Hi Rp,

    We can't make distributable explicitly and distributable meaning is the server distribute the tests on both client and supported machines (here DUT connected to client machine only) and even support machine also installed the HCK Client software.

    Any way put hold for a while (Distributable concept)

    So coming to you question:

    1. First connect all 4 machines to server using the switch or some thing else but make sure that all are same network.

    2. All 4 machines have the HCK Client software.

    3. Now in HCK Studio at Selection tab, first select the machine pool( The pool contain all 4 machines) and select the DUT that connected to machine..

    In this situation you can select which machine out of the 4 machines..?

    Case1: If you select any one out of 4 machines you can able to run the tests on that machine only.

    Case2: If you select the more than one machine see what will happen and tell the response.


    Yesubabu Gude

    Wednesday, June 26, 2013 4:55 AM
  • Hi YesuBaba Gude and ingenico_protocol,

    I have tried adding two machines (win8 and win8 .1) to the machine pool in a 9431 HCK (released recently ).I have also installed the client version of 9431 HCK on both the machines.I and testing a HID device on both .I can see the duplicated fields for both the devices and even the test cases are duplicated in the "Tests" tab.I am able to run the tests on both the machines simluataneously.

    There is no extra effort to make the system Distributable as you said earlier , IMHO.

    Attaching the snapshots for the HCK tests.

    Also m/c names are WIN(win 8.1)  and T (win 8)

    You can see two "HID Sensor Collection"

    Final Conclusion : With HCK 9431  you can run the similair tests on more than two(currently tested with two) machines simultaneously whithout doing any extra setting.

    PLease comment !!

    Rgds,

    Rp


    • Edited by hcktester Thursday, June 27, 2013 4:15 AM Added Picture
    Thursday, June 27, 2013 4:14 AM
  • Hi,

    Yeah I am already pointed above but I thought its not recommended way... ( see the previous reply as below..

    (3) If you select the both you will get repeated tests in tests tab(if both clients have same device ie device selected in selection tab of HCK Studio) ... I think this is default by design for the repeated tests.

    May be in this case we can run tests on both clients (client1 and client2) and this is not the recommended way.))

    Okay come to pint ....This behavior is by design. If you selected multiple devices that you want to be certified, the Windows HCK uses a process of feature query and requirement mapping to apply all jobs that match each device.

    But what I believed before its not the best way of doing because getting duplicated tests for each client and anyway it seems to be fine now.... but in case of the 4 machines of the clients getting 4 copies right ..!!

    Thanks to Rp for giving final conclusion and also thanks very much to ingenico_protocol, and have a nice day...:)


    Yesubabu Gude

    Thursday, June 27, 2013 4:47 AM
  • Yeah, I told you so. Did you try to run the test to see what's happening ?
    Thursday, June 27, 2013 1:03 PM
  • Hi,

    Distibuted Test scheduling Feature on new HCK is for using multiple clients with same OS and same products. If you prepare these multiple clients, HCK can merge same test items on each client to one.  If you still find duplicated test items, please check if you attach test devices with same specification and names.

     For example, I can see duplicated test items with storage device on test device if I set diffrent drive name or different drive number. I recommend you to prepare same environment on multiple clients for distributed testing. Thanks.

    YoYamada   

    Friday, June 28, 2013 5:33 AM
  • Please take a look at the HCK user guide (Click Help button on HCK Studio) or on MSDN here - http://msdn.microsoft.com/en-us/library/windows/hardware/jj124227.aspx

    For distributing tests, take a look at this link http://msdn.microsoft.com/en-us/library/windows/hardware/hh998511.aspx

    Thanks!

     
    Friday, June 28, 2013 7:10 PM