Troubles with LoadTest on remote TFS agent

Question

I created LoadTest with 4 test in TextMix. It runs perfectly when launched locally. However, if I try to run it on TFS agent via build definition, I got the following troubles:

1. Result is "green checkmark + red cross".

I had all four tests in the test plan with loadtest, so I get "4/5 test(s) passed, 0 failed, 0 inconclusive", "State: 4 Passed, 1 None". In the local run I got "5/5".

Why loadtest is not marked as passed on agent run?

2. If I have "agent counters set" in load test, I got "The performance counter category 'Memory' cannot be accessed on computer 'COMPONENTS.ALL HOSTS.WINDOWS 7 SP1 X86 PERF (4).TFSBS-VM-TA-19' (The network path was not found)". Computer named "TFSBS-VM-TA-19" is on the same network and visible and ping'able. I guess the problem is that TFS tries to use long name (with "COMPONENTS.ALL HOSTS...").

How to turn TFS to use short name? (real computer's network name)

3. When I watch (in Visual Studio) results of local test run,  I see test running time as "16 seconds" (if choose "Run duration = 16") or "7 seconds" (if I choose "Use test duration", one of test has Sleep(7000)). I also see all four tests in summary's "Test Results" and in "Top 5".

Remote test running time is always 0 seconds, and no test in summary and in "Top 5 Slowest Tests" at all! The same tests are all "Passed" if run outside of LoadTest TestMix. I tryed different TestMix types (number of tests, sequential) and duration/iteration settings with the same result.

Why my LoadTest has no tests on remote run?

Answer 1

Hi Basil22,

Thank you for your post.

There are two points for you to refer: possible reason for this issue, and recommend tool to run load test.

I suppose you use team build run load test, but in buile result do not see MSTest run the load test right? If I have misunderstood anything, please free feel to let me know. It may relate to the setting in build definition--Process--Basic--Automated Tests. Whether you choose Test assembly file specification(recommended)? Please select Test metadata file(.vsmdi) to run load test, and remember to create a test list in the vsmdi file.

What's more, to run load test, i recommend you use vs test, or VS lab management. Use team build run load test costs lots of time, and you may not see some real-time information.

Best regards,

---

Lily Wu [MSFT]
MSDN Community Support | Feedback to us

Answer 2

Hi, thank you for your attention.

> I suppose you use team build run load test, but in buile result do not see MSTest run the load test right?

Not exactly. We use one Build Definition to build tests and second Build Definition, based on "LabDefaultTemplate.xaml", to run these tests. Your approach is not suitable for us because all we need is executing load-test on a separate remote machine (lab agent).

So, the main problem and our work stopper is:

Test cases from the load-test-mix are not being executed (but being executed in local run or in remote run outside of load test scenario).

Would you give us some thoughts on the issue?

Answer 3

Hi,

I guess the machine you mentioned is in a domain. Would you apart and rejoin the domain then retry?  This should take care of machine name not recognized issue.

On the other hand, would you enable log about Test Controller, Test Agent and Agent process? Following below strategy:

For Controller Service/Agent Service/Agent Process, you need the following sections in the corresponding .config file (qtcontroller.exe.config, qtagentservice.exe.config, qtagent.exe.config, qtagent32.exe.config which by default are located in C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE):

• Inside the <appSettings> section:

  <add key="CreateTraceListener" value="yes"/>

• Inside the <configuration> section (note: “Verbose” is equivalent to “4”):

  <system.diagnostics>
    <switches>
      <add name="EqtTraceLevel" value="Verbose" />
    </switches>
  </system.diagnostics>

• Trace files (by default these are created in the same directory as where controller/agent service/process is located, C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE):
  • Controller: vsttcontroller.log
  • Agent Service: vsttagent.log
  • Agent Process: VSTTAgentProcess.log

Please share the logs online if you need help in analyze the logs. So people can help analyze.

best regards,

---

Forrest Guo | MSDN Community Support | Feedback to manager

Answer 4

Well, there are these logs:

http://download17.files.mail.ru/VC997I/b6727ea18354ba6bc8fe19998c07a329/loadtest_log.zip

(http://files.mail.ru/VC997I if direct link doesn't work; there are loadtest_log.zip: archive with ogs, .loadtest and .cs files)

Testing request was at 17:09.

Load test is named "MyLoadTest3". It has one test "FooTestGood". Also test plan has two other tests: FooTest3 and FooTest4, which are not included in load test. Both tests run normally.

Any ideas why tests didn't run within load test scenario?

Answer 5

Hi,

Thank you sharing the logs. Go through these logs, I find a few issues on your environment setup:

1. On test controller(log: QTController.exe.log), Does not have 'AgentServiceUser Or ConnectTestAgentToTestController' permissions.  Please make sure the Visual Studio, TestController, Test Agent could communicate with each other(pingable).  Please make sure VS Test menu, Manage Test Controller shows up agents well.  This relates to network.
2. QTController.exe.log.  Checking if user KL\DanilaTfsSevice is in Admin Group, returns "User not in the Admin group."  Please add the service account to admin group;
3. QTAgentService.exe.log also clearly shows test agent could not connect to test controller.

Hope this gets you started.

best regards,

---

Forrest Guo | MSDN Community Support | Feedback to manager

Answer 6

Hi,

How is the issue going?  Feel free to reply if you have follow up question.

best regards,

---

Forrest Guo | MSDN Community Support | Feedback to manager

Answer 7

> How is the issue going?  Feel free to reply if you have follow up question.

We had May-holidays in Russia so I cannot answer quickly :)

Now there are my status update:

1.

a) I don't fully understand what permission I should add.

b) Controller and Agent can ping each others name. VS test menu cannot access controller/agents because we use TFS for launching tests, and controller cannot be in TFS and be available for VS test menu simultaneously. However, if we detach controller from TFS, and run tests on it using mstest.exe, tests are run perfectly.

c) Agent's name is "COMPONENTS.ALL HOSTS._WINDOWS 7 SP1 X86.TFSBS-VM-TA-13", is it okey? I ping just "TFSBS-VM-TA-13", not a full name.

2. I added KL\DanilaTfsSevice to Admins but nothing changed.

3. I tryed with new Agent who has no problem with connection (at least I see nothing like previous error in log)

New logs are there:

http://download17.files.mail.ru/BBW4TA/52d2fb9547064d3fdd24fff3cd4bcbb5/logs2.zip

Unfortunatelly, result is the same - no tests are run in LoadTest scenario.

Answer 8

Hi,

I guess either Visual Studio client, test agent, or test controller is in workgroup, but not in domain.  Please make sure you can connect test agent, test controller on Visual Studio client through Remotet Desktop. Sometimes, you can ping a computer, but real IP address is different from what you see in ping.  This is very key.

I searched around, and it turns out 'AgentServiceUser Or ConnectTestAgentToTestController'  permission may not be accurate permission. On test controller, please make sure you account is in TeamTestAgentService group, TeamTestControllerAdmins, and TeamTestControllerUsers group.

What if you run the tests remotely in Visual Studio, detached TFS project collection from Test controller?

btw, I could not get new logs, please reupload.

thanks and regards,

---

Forrest Guo | MSDN Community Support | Feedback to manager

Answer 9

Hi,

1. I can access both test agent and controller via Remote Desktop.

2. I added agent user to TeamTestControllerAdmins and TeamTestControllerUsers (it already was in TeamTestAgentService) but nothing changed.

3. If I run tests on detached Test Controller (via mstest.exe) tests run perfectly.

Moreover, I discovered that tests are run perfectly in... physical environment! If I use LabTemplate.xaml and try to launch my load test scenario on Lab Environmebt, they didn't run. But if I use custom xaml and try launch my load test scenario on Physical Environment (using the same computer), they run! So we have troubles only with Lab Environment.

P.S. Sorry for bad link, logs are there:
http://files.mail.ru/BBW4TA  (click on logs2.zip)

Answer 10

Hi,

I reviewed the latest log and there're two issues, both in QTController.exe.log:

1.  DIA initialization threw: COMException (0x80040154): Retrieving the COM class factory for component with CLSID {B86AE24D-BF2F-4AC9-B5A2-34B14E4CE11D}
2. LoadTestResultsCollector: Reporting exception: Microsoft.VisualStudio.TestTools.WebStress.LoadTestCounterCategoryNotFoundException: The performance counter category 'Memory' cannot be accessed on computer 'COMPONENTS.ALL HOSTS._WINDOWS 7 SP1 X86.TFSBS-VM-TA-13' (The network path was not found) ; check that the category and computer names are correct.

To resolve the first issue, please run register command:

    regsvr32 C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\Packages\Debugger\msdia100.dll

To resolve second issue, please double check the computer name.  VM name and VM computer name are different value sometimes.  Computer name length should be restrict.

best regards,

---

Forrest Guo | MSDN Community Support | Feedback to manager