locked
Limit on concurrent test agent runs? RRS feed

  • Question

  • I've noticed there seems to be a limit to 10 concurrent test agents running, is this a hard limit or something we can set?
    shaun mchugh
    • Moved by Mathew Aniyan MSFT Saturday, June 25, 2011 8:58 AM Question on Test Infrastructure (From:Visual Studio UI Automation Testing (includes CodedUI))
    Wednesday, June 22, 2011 4:15 PM

All replies

  • not sure why this was moved to unit testing as this is with codedui tests and the MS test agent / test controller.

    shaun mchugh
    Saturday, June 25, 2011 2:33 PM
  • Hi Shaun mchugh,

     

    Not sure if I understood your question correctly. Are you asking about the number of agents that can be managed by a test controller at a given point of time ?

    If so, there is no hard limit on the number of agents that has to be registered under a test controller. It depends on your need and the hardware configuration of your controller. For normal load testing scenarios, you can typically have up to 50 test agents under a test controller.

    Ping us back if that doesn't answer your question.

    Thanks

    Bhuva

    Monday, June 27, 2011 11:31 PM
  • Hi Bhuva,

     

    We have around 16 computers set up to run the CodedUI Automation tests using the MS Test agent on each machine connecting to 1 Test Controller. What we are finding is that once we have 10 agents running UI tests any further runs we attempt to start hang. 


    shaun mchugh
    Tuesday, June 28, 2011 2:58 PM
  • Hi there,

     

    we're having the exact same issue. We have 15 machines, each one has a test agent, and all of them connect to the same test controller. The first 10 machines test execution starts, and the remaining five stay in state "Waiting for test controller".

     

    Later we can also see that, as soon as one of the first machines test execution ends, one of the "waiting" machines starts its execution.

     

    Is there any configuration option we're missing? Shaun have you already found out what happened in your scenario?

     

    Thanks,

    Telmo

    Wednesday, August 3, 2011 4:36 PM
  • Hi Telmo,

    I haven't been able to come across a workaround other than adding another test controller, I'm told its resource based but it seems to me to be a hard limit(especially now that you've hit the 10 limit as well), there's not much information about this either way.


    shaun mchugh
    Wednesday, August 3, 2011 4:39 PM
  • Also forum moderators can we move this thread back to the CodedUI forum as this isn't related to unit testing.
    shaun mchugh
    Wednesday, August 3, 2011 4:40 PM
  • I'm not sure , but after some further investigations, I could see this in the qtcontroller log:

     

    I, 7760, 5, 2011/08/03, 18:20:44.403, SERVERTEAM03\QTController.exe, RunMonitor: ThreadPoolSize: 10

    I'd say the problem is that threadpoolsize...

     

    V, 7760, 12, 2011/08/03, 18:40:14.494, SERVERTEAM03\QTController.exe, RunMonitor: Executing query SELECT * FROM TestRun WHERE (State = 'InProgress' OR  State = 'Waiting') AND IsAutomated = 1 AND TestEnvironmentId IN ('a0de76c6-25ab-4ef6-bafa-11e232c882e0','ab139a50-221d-47eb-afe1-a5748ffb69b9','afd5de0f-6e5d-482c-abcc-81f31ef50e38','54a19bb2-e55c-4fed-805e-678298790877','3bf20a1f-bcba-47b0-ab04-40f5a2c67af6') ORDER BY CreationDate

    I, 7760, 12, 2011/08/03, 18:40:14.513, SERVERTEAM03\QTController.exe, RunMonitor: Queueing run 1096 against environment ab139a50-221d-47eb-afe1-a5748ffb69b9
    I, 7760, 12, 2011/08/03, 18:40:14.513, SERVERTEAM03\QTController.exe, RunMonitor: Added run [1096] to active run list for environment ab139a50-221d-47eb-afe1-a5748ffb69b9
    I, 7760, 45, 2011/08/03, 18:40:14.517, SERVERTEAM03\QTController.exe, RunMonitor: Starting run 1096
    I, 7760, 45, 2011/08/03, 18:40:14.517, SERVERTEAM03\QTController.exe, TcmRunner: Preparing to execute test run [1096].

    I, 7760, 12, 2011/08/03, 18:40:14.524, SERVERTEAM03\QTController.exe, RunMonitor: Queueing run 1097 against environment afd5de0f-6e5d-482c-abcc-81f31ef50e38
    I, 7760, 12, 2011/08/03, 18:40:14.524, SERVERTEAM03\QTController.exe, RunMonitor: Added run [1097] to active run list for environment afd5de0f-6e5d-482c-abcc-81f31ef50e38

    I, 7760, 12, 2011/08/03, 18:40:14.545, SERVERTEAM03\QTController.exe, RunMonitor: Queueing run 1098 against environment 3bf20a1f-bcba-47b0-ab04-40f5a2c67af6
    I, 7760, 12, 2011/08/03, 18:40:14.546, SERVERTEAM03\QTController.exe, RunMonitor: Added run [1098] to active run list for environment 3bf20a1f-bcba-47b0-ab04-40f5a2c67af6

    1096 was the tenth test, it started running, and then 1097, 1098,... remained queued.

     

    Can anyone confirm this? And is it possible to increase that value? Or is is mandatory to use two test controllers?

     

    Thanks,

    Telmo

    • Proposed as answer by talberto Thursday, October 20, 2011 2:33 PM
    • Unproposed as answer by talberto Thursday, October 20, 2011 2:33 PM
    Wednesday, August 3, 2011 6:10 PM
  • Just for the record, we've solved this today.

     

    Going to QtController.exe.config, there's a parameter called MaxConcurrentTcmRuns

    <add key="MaxConcurrentTcmRuns" value="13" />

     

    Changing it to 13 worked, now all the machines start the tests at the same time.

    • Proposed as answer by talberto Thursday, October 20, 2011 2:33 PM
    Thursday, October 20, 2011 2:33 PM
  • I have set up a physical environment with 4 agents and one controller.I have 9 test cases for execution.

    When i start the Run from the Labmanger.

    each agent is executing one test cases. So at a time 4 test cases are getting executed and rest of the test cases are queued .But once  4 test cases are getting executed completly.The second set of test cases are sittilng in queued status for long time (5 to 7 mins) and the agent is showing as still running in progress.After waiting for 5 to 7 mins the agent is executing the second set of test cases.

    Can you please help here how can i make the agent continously execute the test cases so that  the run will complete fast.

     

    Below are myappseting in config file of controller

     <appSettings>
        <add key="LogSizeLimitInMegs" value="20"/>
        <add key="AgentConnectionTimeoutInSeconds" value="120"/>
        <add key="AgentSyncTimeoutInSeconds" value="300"/>
    <add key="MinimumTestsPerAgent" value="1"/> 
        <add key="MaxConcurrentTcmRuns" value="4"/>
        <add key="ControllerServicePort" value="6901"/>
        <add key="ControllerUsersGroup" value="TeamTestControllerUsers"/>
        <add key="ControllerAdminsGroup" value="TeamTestControllerAdmins"/>
        <add key="CreateTraceListener" value="yes"/>
      </appSettings>

     

    Friday, November 11, 2011 1:26 AM