Very low performance locally when running Unit Tests

Question

Hi,

I'm working in a quite large TDD-project where Unit Testing is a central part of the daily development. We have about 4500 Unit Tests in our project and the strange thing is that on two machines the a testrun of all Unit Tests runs about 4 times as slow than on the other machines, this applies both when running the tests by MSTest as well as ReSharper. We've tried to go through the settings in order to find some setting that differ among the machines, but we haven't found any differences in local.testsettings or through the Test-setting s GUI.

We also tried to "force" Visual Studio to run 4 tests simultaneously by setting the parallelTestCount to "0" and "4", however this doesn't improve the results.

All developers use the same hardware except for me and the other person with slow unit tests. The difference is that our machines have 4GB RAM more installed than the other machines. Some of my machine spec:

Intel Xeon (Quad, x64, 3.2 GHz), 12 GB ram, W7 Ultimate,  Visual Studio 2010 (TFS PowerTools, ReSharper 5.1)

When running the Unit Tests the processors peak around 30% of total capacity and the RAM uses 50% (that is, 6GB) constantly. Does anyone encounter any similar problem and how did you solve it?

Thanks in advance!

Answer 1

Hi zalle,
 
Thank you for your question.

I am trying to involve someone familiar with this topic to further look at this issue. There might be some time delay. Appreciate your patience.
 
Thank you for your understanding and support.

---

Tony Xiao [MSFT]
MSDN Community Support | Feedback to us

Answer 2

Hi Tony,

Thanks, I appreciate all help I can get!

/zalle

Answer 3

Hi Zalle,

Run Unit Tests Faster Using a Computer with Multiple CPUs or Cores by using parallelTestCount parameter.  Please refer to the next link.

http://msdn.microsoft.com/en-us/library/ee921484.aspx

Thanks!

---

Sophia Lu [MSFT]

Answer 4

Hi Sophia,

As I mentioned in my original post I already tried to change the parallelTestCount-parameter, both with explicitly specifying how many cores I would like to use as well as allowing VS to use as many cores as possible. However, this neither of these settings improved the performance positively. Do you have any other idea on how to resolve the issue?

Thanks!

/zalle

Answer 5

Hi Zalle,

According to your description, the problematic machines are 4G RAM more than the good ones, right. 

Intel Xeon (Quad, x64, 3.2 GHz), 12 GB ram, W7 Ultimate,  Visual Studio 2010 (TFS PowerTools, ReSharper 5.1)

When running the Unit Tests the processors peak around 30% of total capacity and the RAM uses 50% (that is, 6GB) constantly.

How is the data when running the Unit Test on the good machines?

Thanks!

---

Sophia Lu [MSFT]

Answer 6

Hi Sophia,

I apologize for the late reply; I missed the notification message for the thread update.

Yes, you are correct, the unit test problem occurs on the machines with more hardware installed. I have made some new measurements comparing the two machines (using the Resource Monitor on QTAgent32 process in Windows 7) while running the Unit Test suite with 4840 tests. Here are the results:

Problem-free machine:

CPU: Peaks at 60% with an average of ~20%
RAM: Steady number of 55% (Hard faults/s: peaks at 87 with an average of ~75)
Test suite execution time: 1m 23s

Problematic machine (with default parallelTestCount):

CPU: Peaks at 35%, with an average of ~25.4%
RAM: Steady number of 35% (Hard faults/s: peaks at 5 with an average of average 1)
Test suite execution time: 9m 3s

Problematic machine (with parallelTestCount = 4):

CPU: Peaks at 46% with an average of 25.8%
RAM: Steady number of 36% (25% when re-booted) (Hard faults/s: peaks at 16 with an average of 4)
Test suite execution time: 9m 0s (8m 54s when re-booted)

As you can see in the above performance measures, the parallelTestCount-parameter (as found in: http://blogs.msdn.com/b/vstsqualitytools/archive/2009/12/01/executing-unit-tests-in-parallel-on-a-multi-cpu-core-machine.aspx) doesn’t seem to affect the overall performance of the Unit Tests and I have no idea what the problem could be. Do you have any idea?

Thanks in advance!

/zalle