none
Very low performance locally when running Unit Tests

    Întrebare

  • Hi,

    I'm working in a quite large TDD-project where Unit Testing is a central part of the daily development. We have about 4500 Unit Tests in our project and the strange thing is that on two machines the a testrun of all Unit Tests runs about 4 times as slow than on the other machines, this applies both when running the tests by MSTest as well as ReSharper. We've tried to go through the settings in order to find some setting that differ among the machines, but we haven't found any differences in local.testsettings or through the Test-setting s GUI.

    We also tried to "force" Visual Studio to run 4 tests simultaneously by setting the parallelTestCount to "0" and "4", however this doesn't improve the results.

    All developers use the same hardware except for me and the other person with slow unit tests. The difference is that our machines have 4GB RAM more installed than the other machines. Some of my machine spec:

    Intel Xeon (Quad, x64, 3.2 GHz), 12 GB ram, W7 Ultimate,  Visual Studio 2010 (TFS PowerTools, ReSharper 5.1)

    When running the Unit Tests the processors peak around 30% of total capacity and the RAM uses 50% (that is, 6GB) constantly. Does anyone encounter any similar problem and how did you solve it?

    Thanks in advance!

    25 aprilie 2012 07:20

Toate mesajele

  • Hi zalle,
     
    Thank you for your question.

    I am trying to involve someone familiar with this topic to further look at this issue. There might be some time delay. Appreciate your patience.
     
    Thank you for your understanding and support.


    Tony Xiao [MSFT]
    MSDN Community Support | Feedback to us

    26 aprilie 2012 07:54
    Moderator
  • Hi Tony,

    Thanks, I appreciate all help I can get!

    /zalle

    2 mai 2012 11:59
  • Hi Zalle,

    Run Unit Tests Faster Using a Computer with Multiple CPUs or Cores by using parallelTestCount parameter.  Please refer to the next link.

    http://msdn.microsoft.com/en-us/library/ee921484.aspx

    Thanks!


    Sophia Lu [MSFT]

    21 mai 2012 05:37
  • Hi Sophia,

    As I mentioned in my original post I already tried to change the parallelTestCount-parameter, both with explicitly specifying how many cores I would like to use as well as allowing VS to use as many cores as possible. However, this neither of these settings improved the performance positively. Do you have any other idea on how to resolve the issue?

    Thanks!

    /zalle

    24 mai 2012 10:29
  • Hi Zalle,

    According to your description, the problematic machines are 4G RAM more than the good ones, right. 

    Intel Xeon (Quad, x64, 3.2 GHz), 12 GB ram, W7 Ultimate,  Visual Studio 2010 (TFS PowerTools, ReSharper 5.1)

    When running the Unit Tests the processors peak around 30% of total capacity and the RAM uses 50% (that is, 6GB) constantly.

    How is the data when running the Unit Test on the good machines?

    Thanks!


    Sophia Lu [MSFT]

    6 iunie 2012 09:54
  • Hi Sophia,

    I apologize for the late reply; I missed the notification message for the thread update.

    Yes, you are correct, the unit test problem occurs on the machines with more hardware installed. I have made some new measurements comparing the two machines (using the Resource Monitor on QTAgent32 process in Windows 7) while running the Unit Test suite with 4840 tests. Here are the results:

    Problem-free machine:

    CPU: Peaks at 60% with an average of ~20%
    RAM: Steady number of 55% (Hard faults/s: peaks at 87 with an average of ~75)
    Test suite execution time: 1m 23s

    Problematic machine (with default parallelTestCount):

    CPU: Peaks at 35%, with an average of ~25.4%
    RAM: Steady number of 35% (Hard faults/s: peaks at 5 with an average of average 1)
    Test suite execution time: 9m 3s

    Problematic machine (with parallelTestCount = 4):

    CPU: Peaks at 46% with an average of 25.8%
    RAM: Steady number of 36% (25% when re-booted) (Hard faults/s: peaks at 16 with an average of 4)
    Test suite execution time: 9m 0s (8m 54s when re-booted)

    As you can see in the above performance measures, the parallelTestCount-parameter (as found in: http://blogs.msdn.com/b/vstsqualitytools/archive/2009/12/01/executing-unit-tests-in-parallel-on-a-multi-cpu-core-machine.aspx) doesn’t seem to affect the overall performance of the Unit Tests and I have no idea what the problem could be. Do you have any idea?

    Thanks in advance!

    /zalle

    28 iunie 2012 11:08