none
Load Test: virtual users configuration problem RRS feed

  • Question

  • Hello everybody,

    I have a problem in settings of load test. Here is some settings of my load test:

    - Load Pattern: Constant (2 users)

    - Test mix model: Based on the number of virtual users

    - Test mix: 2 unit tests

    - Run settings: Use specificed Iterations, Test iterations:4 (i want each virtual user run the set of tests, or each unit test is executed 2 times, so number of iterations = 2x2).

    The project is compiled successfully, I run the load test many times but the results are not what i expect: the first unit test is executed 3 times and the second is executed just 1 time. I ran the test 10 times, only 2 times with result test 1: 2 times, test 2: 2 times.

    Is it an error of VS2012,MSTest or did i make mistakes?

    Thank you for your support.

    Long Bien

    Friday, March 1, 2013 10:44 AM

Answers

  • Hello Biennl86,

    In my load test, i choose test mix based on sequential order with a constant of user (2). The load test has 2 unit test, number of iterations = 4. But the result is always 3:1.

    I can’t repro your issue with my scenario based on your settings. As my understanding, it the test mix is based on sequential order, the result should be 2:2.

    If you get this result with some new tests, I doubt maybe the issue is related to your VS. I suggest that you could locate to the IDE folder then run these commands to check if it can help:

    C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE

    Devenv.exe /SafeMode: Launches the IDE in safe mode loading minimal windows.

    Devenv.exe /ResetSettings: Restores the IDE's default settings, optionally resets to the specified VSSettings file.

    Devenv.exe /ResetAddin: Removes commands and command UI associated with the specified Add-in.

    Devenv.exe /ResetSkipPkgs: Clears all SkipLoading tags added to VSPackages.

    After these, you can create a new project in a new solution to avoid if there’re any problems of the old settings. If still has no help, how about repairing your VS?

    If still no help, you can provide us a simple sample in order to make us repro your issue. Please attach your project/solution, you can upload it to the sky drive, and then share the download link in your post. We try to check it with our computers.

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us
    Develop and promote your apps in Windows Store
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Wednesday, March 6, 2013 2:03 AM
    Moderator

All replies

  • Hello Biennl86,

    Thank you for your post.

    Based on the following article, this model of test mix determines the percentage of virtual users who will run a particular test. If you use this model of test mix, the next test to start is based not only on the assigned percentages but also on the percentage of virtual users who are currently running a particular test. At any point in the load test, the number of users who are running a particular test matches the assigned distribution as closely as possible.

    Reference: http://msdn.microsoft.com/en-us/library/bb514186.aspx#PercentageBasedonVirtualUsers

    I suggest that you can assign the percentage of two unit test to 50% and 50% and then run the load test to check if you can get the expected result.

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us
    Develop and promote your apps in Windows Store
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.


    Monday, March 4, 2013 3:17 AM
    Moderator
  • Hello Amanda Zhu,

    Thank for your reply.

    I didn't get the expected result when i run the load test in VS2012. The distribution of tests is always 3:1, not 2:2 as expected (I ran this test 10 times, there was only 2 times that i had the expected result). My tests are based on a sequential order and in the link above, it marked well: "Selecting the Based on sequential test order option makes each virtual user run all the tests in the scenario in the order that the tests were defined."

    I don't understand. Any other suggest? It's really urgent. Thanks in advance.


    Monday, March 4, 2013 7:08 PM
  • Hello,

    I didn't get the expected result when i run the load test in VS2012. The distribution of tests is always 3:1, not 2:2 as expected (I ran this test 10 times, there was only 2 times that i had the expected result).

    When you get this situation, I would like to know which test mix model you are using. Test mix percentage based on the number of virtual users starting tests or test mix based on sequential test order?

    If you use the former, this model of test mix determines the percentage of virtual users who will run a particular test. The next test to start is based not only on the assigned percentages but also on the percentage of virtual users who are currently running a particular test. It is possible that the distribution of tests is not always 2:2 when test iteration is 4 even though that you set the assigned percentages of tests is 50% and 50%.

    If you use the later, each virtual user will run all tests in the scenario in the order. Then the distribution of tests should be 2:2 when you set the test iteration is 4.

    If you want to get expected result, I think that you can choose to use test mix based on sequential test order.

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us
    Develop and promote your apps in Windows Store
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Tuesday, March 5, 2013 1:21 AM
    Moderator
  • Hi Amanda Zhu,

    Thanks for your reply.

    In my load test, i choose test mix based on sequential order with a constant of user (2). The load test has 2 unit test, number of iterations = 4. But the result is always 3:1. :(. It's really bizarre!! I did it many times (create new test, resettings,etc...) but no chance..

    Tuesday, March 5, 2013 10:41 AM
  • Hello Biennl86,

    In my load test, i choose test mix based on sequential order with a constant of user (2). The load test has 2 unit test, number of iterations = 4. But the result is always 3:1.

    I can’t repro your issue with my scenario based on your settings. As my understanding, it the test mix is based on sequential order, the result should be 2:2.

    If you get this result with some new tests, I doubt maybe the issue is related to your VS. I suggest that you could locate to the IDE folder then run these commands to check if it can help:

    C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE

    Devenv.exe /SafeMode: Launches the IDE in safe mode loading minimal windows.

    Devenv.exe /ResetSettings: Restores the IDE's default settings, optionally resets to the specified VSSettings file.

    Devenv.exe /ResetAddin: Removes commands and command UI associated with the specified Add-in.

    Devenv.exe /ResetSkipPkgs: Clears all SkipLoading tags added to VSPackages.

    After these, you can create a new project in a new solution to avoid if there’re any problems of the old settings. If still has no help, how about repairing your VS?

    If still no help, you can provide us a simple sample in order to make us repro your issue. Please attach your project/solution, you can upload it to the sky drive, and then share the download link in your post. We try to check it with our computers.

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us
    Develop and promote your apps in Windows Store
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Wednesday, March 6, 2013 2:03 AM
    Moderator