none
Are Visual Studio Web Performance load tests accurate?

    Question

  • Visual Studio 2013 Ultimate
    Scenario: 
    Set up Web Performance test consisting of browsing to my intranet home page.
    Set up Load Test running the one Web Performance test - constant 100 user load for 10 minutes.
    Test Mix Model:  Based on the number of virtual users.  (The options for Test Mix model confuse me)
    The test is run from one workstation (no test rig).

    The Avg. Page Response time reported is poor, around 45 seconds

    I question the validity of the test because while the test is running, I can bring the intranet home page in my browser in about 3 seconds --  on the same computer that is running Visual Studio, as well as from other computers.

    Does this demonstrate that Visual Studio is not working correctly?
    Am I doing this wrong?
    Any advice is appreciated.





    • Edited by bankdog Tuesday, November 15, 2016 9:51 PM
    Tuesday, November 15, 2016 9:50 PM

All replies

  • Hi bankdog,

    The Web Performance Test is a little difference with open the URL in browser directly.

    In Web Performance Test, exclude record the URL that you access in browser, there are most dependent requests. These dependent requests are accessed in Web Performance Test one by one which will take some times.

    But in browser, these dependent requests often send asynchronous. The browser will load the main content of the page, and other content will request through these requests on background.

    And in Web Performance Test, every request has a Think Time property, which used to emulate users thinking something when access the website.

    Best Regards,
    Weiwei


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Wednesday, November 16, 2016 9:50 AM
    Moderator
  • thanks for the response, but it seems that you are saying that the Visual Studio load test of 100 users does NOT accurately report the experience that 100 users accessing the site separately from browsers would experience.   This is disappointing, as I was counting on Visual Studio load testing to provide value.

    Are you saying that if I deleted the dependent requests that it would be more realistic?

    I need to get a true and accurate reflection of how the website would perform under load.  Am I using the wrong tool?

    Thanks...

    Wednesday, November 16, 2016 11:53 AM
  • Hi bankdog,

    >> but it seems that you are saying that the Visual Studio load test of 100 users does NOT accurately report the experience that 100 users accessing the site separately from browsers would experience.

    I mean that the Load Test will emulate the real situation that a user access a website. In real world, when we access a page, there are different Influence factors affect the access process. For example network, user operation process.

    And Load Test also used to test your machine environment, so it also affected by machine environment, such as CPU, Memory and so on. This is why the Avg. Response Time in Load Test result will more than the actually result.

    So the Load Test record a true and accurate reflection of how the website would performance under load in a real world.

    If you want to compare the page response time with your operation, please run your Web Performance Test and set "Follow Redirects" as "False" and set "Think Time(second)" as "0". But I think your operation that access the URL direct in browser without any influence factors is a perfect state, not a real world situation.

    Best Regards,
    Weiwei


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Thursday, November 17, 2016 2:55 AM
    Moderator