Web and Load Test FAQs RRS feed

  • General discussion

  • Q: Where can I find more information about Web and Load Testing?

    A: Links to info on web and load tests


    Q: How can I record AJAX requests, or requests made from popups using the DoModalDialog?

    A: Upgrade to Visual Studio 2008, or use the recorder in fiddler. The Visual Studio 2005 Web test recorder cannot capture requests from ActiveX controls, AJAX requests from java script, or pop-up window requests. This was fixed in Visual Studio 2008.


    Q: What does the Parse Dependent Requests property on requests do?

    A: By default, when a Web test is executed, HTML responses received from the Web server are parsed and any dependent requests, such as images or style sheets, are automatically submitted. This is usually desirable because it places the most realistic load on the server.

    However, if the goal of your load test is to place the maximum load on your application server as opposed to your Web server, you might wish to disable this behavior by setting the ParseDependentRequests properties on all of your Web test requests to false. If you do this, only the top level request is fetched, but not the images, style sheets, and so forth. 


    You can also selectively block certain dependent requests.

    1. Set Parse Dependent Requests to false.
    2. Manually add the dependent links back onto the page by right-clicking on the request and selecting Add Dependent Request.

    Q: How can I test Web services?

    A: Web services can be tested in a Web test or a unit test. To test a Web service in a Web test, read Create a Web Service Test. To test a Web service in a unit test, read Load Testing Web Services with Unit Tests.


    Q: How is data binding handled in Web and load tests?

    A: Visual Studio Team System loads data sources at the start of a test run. Therefore, any updates to the data records in the data sources during test execution will not be picked up by the test.


    To add different data sources from the Web test editor, read the blog post Adding Different Types of Data Sources to a Web Test.


    To create custom data binding, read the blog post Custom Data Binding in Web Tests.

    For more information, see About Data Binding in Web Tests.


    Q: What can extraction and validation rules do?

    A:Extraction rules are useful when you have a piece of data from one page that is needed on another page. For example, extraction rules can be used to extract information from responses, such as http header, attribute value, form field, hidden field, text, and regular expression. The extracted values can then be added to the Web test context for future usage. For general information on Extraction rules, see About Extraction Rules. To try the feature, see Walkthrough: Adding Validation and Extraction Rules to a Web Test. To write a custom extraction rule for your test, read How to: Create a Custom Extraction Rule and Custom ExtractionRule to extract form fields by index.


    Validation rules are useful when you want to verify that expected data, such as http tag, attribute, request time and text, appear in the responses or perform Web functional testing. For general information on Validation rules, see About Validation Rules. To try the feature, see Walkthrough: Adding Validation and Extraction Rules to a Web Test. To write a custom validation rule, read How to: Create a Custom Validation Rule. A sample validation rule can be found at A custom ValidationRule to catch redirects to error pages.


    Q: How are view state and hidden fields tracked in a Web test?

    A: Refer to Web Test Authoring and Debugging Techniques, section “Handling View State and Other Dynamic Parameters”.


    Q: Why do I get the error “ Context parameter ‘HIDDEN1._EVENTTARGET’ not found in the test context”?

    A: This was a bug in VS 2005 which is fixed in SP1. Get SP1 or VS 2008.

    When a hidden field is modified by JavaScript in an OnClick event handler, it is possible that automatic hidden field binding will be incorrectly applied. This is a known bug in the release version of Visual Studio 2005.

        <input name=”btnNext” type=”button” value=”Next”

        Onclick=”_doPostBack(‘btnNext’,’’);” />

    With ASP.NET sites, this problem most commonly occurs when a Web control calls the __doPostBack() JavaScript method to set the __EVENTTARGET hidden field as shown above. Automatic hidden field binding results in the form parameter having a value such as {{$HIDDEN1.__EVENTTARGET}}, instead of the actual value — btnNext. To correct this problem, the parameter value must be set to the value being set in JavaScript (for example, btnNext).


    Q: Why do I get the error “ Request Failed: Context parameter ‘HIDDEN1._VIEWSTATE’ not found in the test context” when running in a load test?

    See this link for info. Search for "Web Server Responds Differently During Execution than Recording". The most common reason is that during a load test, the application server redirects to an error page rather than the expected page. Some extraction rules (including hidden field extraction) will not fail the page when this happens, so the error doesn't actually occur in the test until the subsequent page tries to use the extracted value.


    Q: What can Web test/request plug-ins do?

    A:Web test plug-ins provide a way for you to isolate some code outside the main declarative statements in your Web test. A customized Web test plug-in allows you to call some code as the Web test is run. The Web test plug-in is run one time for every test iteration. Code in the PreWebTest handler will be called before the Web test starts issuing requests. Code in the PostWebTest handler will be called after the Web test completes. For example, the cleanup code can be inserted here.


    Web request plug-ins provide a way for you to isolate code outside the main individual requests in your Web test. A request plug-in allows you to call code as each request is run in a Web test. Code in the PreRequestHandler function will be called before a request is issued. For example, you can skip a request if certain condition is met. Code in the PostRequestHandler function can be used to retrieve information from response pages.


    Because you have access to the Web test object in the plug-ins, you can manipulate the Web test context, database cursor, or stop a Web test by adding code in different event handlers. For details about how to create a custom Web test plug-in, read How to: Create a Web Test Plug-In. For details about how to create a custom Web request plug-in, read How to: Create a Web Test Request Plug-In.


    Q: Why is my Web test aborted after five minutes when it is run standalone?

    A: It is possible that you are running into the default test timeout. To change the timeout setting, follow these steps:

    On the Test menu, choose Edit Run Configurations and then select the run configuration file.

    1. Click Test Timeouts
    2. Reset the timeout limits.

    However, when your run the web test in a load test, the test timeout setting for a web test is not enforced.


    Q: How can I use test context and pass data between tests?

    A: To access Web test context in a coded Web test, use this.Context in C#.

    To access Web test context in a rule or Web test/request plug-in, use e.WebTest.Context.

    To access load test context in a load test plug-in, use loadTest.Context in Initialize().


    To access load test context from a Web test or a unit test, add the following code (C# ) to the TestStarting handler in a load test plug-in.

        private void TestStarting(object source, TestStartingEventArgs testStartingEventArgs)
            foreach (KeyValuePair<string, object> keyValuePair in m_loadTest.Context)
                testStartingEventArgs.TestContextProperties.Add(keyValuePair.Key, keyValuePair.Value);

    To access Web/unit test context in a load test plug-in, use the testStartingEventArgs.TestContextProperties in the TestStarting handler.


    To override the value of a Web test context parameter in a load test, add the same context parameter to the load test and reset the value.


    Q: What is the difference between Connection Pool and Connection Per User?

    A: Refer to Advanced Load Testing Features of Visual Studio Team System, section “Choose the Appropriate Connection Pool Model.” For more information, see Considerations for Large Load Tests.


    Q: How many simultaneous connections can a load test make?

    A: Factors that limit the number of simultaneous connections the load test agent can make are:

    1. The operation system that a Web server (IIS) is running on. With Windows XP, the maximum is 10.
    2. The Web server. Some versions of Web servers (IIS) set limit on the maximum number of connections.

    When the number of connection the load agent tries to make exceeds the limit, HttpError 403 will occur and be logged in the Error table in the Load Test Monitor/Analyzer.


    Q: How can I choose the appropriate user load in a load test?

    A: Refer to Advanced Load Testing Features of Visual Studio Team System, section “Choose an appropriate load profile.” For more information, see Considerations for Large Load Tests.


    Q: What are “Think Time”, “Think Time Between Test Iterations”, and “Think Profile”?

    A: Think times are used to simulate human behavior that causes people to wait between interactions with a Web site. Think times occur between requests in a Web test and between test iterations in a load test scenario. Think times of requests are recorded when requests are recorded. Using think times in a load test can be useful in creating more accurate load simulations. You can change whether think times are used or ignored in load tests. The Think Profile is a setting that applies to a scenario in a load test. The setting determines whether the think times saved in the individual Web tests are used during the load test. By default, think time is turned on in a load test. For more information, see About Think Times and Considerations for Large Load Tests.


    Q: How can I collect counters on a remote server in a load test?

    A: You can configure the load test to collect the performance counters on the remote computers by adding mappings in the load test.

    1. Right-click Run Settings and select Manage Counter Sets.
    2. Add the performance counter sets to the load test which you would like to collect data for.
    3. Right-click the counter set to launch the “Pick Performance Counters” dialog, and select counters for the remote server.

    If a test is run remotely, then the controller service account needs to have permission to collect performance counters. If the account is not a member of the Admin or Power User group on the remote computer, then you need to add that user to the "Performance Monitor Users" group. For more information, see How to: Manage Counter Sets.


    Q: What are the common issues associated with counter collection?

    A: The three most common issues associated with counter collection are:

    1. Gaps in the graph. This indicates the controller or the target machines are too busy to collect/provide performance counters.
    2. LoadTestCounterCategoryNotFoundException. The possible reasons for this are the counter category is not installed on the computer, the user account does have permission to collect the counters, or a firewall issue.
    3. Could not find dependent counter needed to apply threshold rule: \\server\LoadTest: Page\Avg. Page Time(_Total). This does not indicate a problem with the load test. The exception comes from one of the default threshold rules. It happens because this rule depends on the Avg. Page Time(_Total) counter which was not collected for the sample period the error occurred.


    For more information, see Troubleshooting Load Tests.


    Q: How is load test execution affected by database access methods?

    A: When you add data source to a Web test, you can choose from three different access options.

    1. Sequential - If you set a data source to sequential, it uses the records from the data source in the order it reads them in. When it reaches to end of the records, it will loop back to the beginning and start again. It will continue to do this for the duration of the load test.
    2. Random - If you select random, then a random row will be used for each test iteration. This will continue to happen for the duration of the load test.
    3. Unique - If you select unique, the load test will access the data sequentially, but it will only execute 1 test iteration for each row in the data source regardless of what the test duration is set to. So if you have 20 rows, then the load test will execute 20 tests and then stop.

    For more information, see About Data Binding in Web Tests.


    Q: How can I control caching in a load test?

    A: The Percentage of New Users under Scenario property affects the way in which the load test run time engine simulates the caching that would be performed by a Web browser. For more information, see Advanced Load Testing Features of Visual Studio Team System, section “Choose a value for the ‘Percentage of New Users’ property.” For more information, see Considerations for Large Load Tests.


    Q: Why is a validation rule in a Web test not executed in a load test?

    A: Execution of validation rules has impact on performance. By default the validation level of a request is set to high. When you create a load test, you can specify what level of validation rules you want to execute under the Run Settings node. By default, it is set to low, therefore only rules which have a level of low will be executed. If you set the validation level in the load test to High, then the load test will execute all rules. For more information, see About Validation Rules.


    Q: How can I limit the number of tests run in a load test?

    A: Please read blog post Using a load test plug-in to limit the number of tests run in a load test.


    Q: How can I clean up or change the load test results repository?

    A: By default, load test results are written into an instance of SQL Express. SQL Express is limited to using a maximum size of 4 GB of disk space.


    If you want to delete old load test runs in the database, read the blog post Deleting Old Load Test Results. If you need a larger database, you should consider configuring the Load Test Results Store to use an instance of the full Microsoft SQL server product. Detailed information on this can be found in Using Results Stores.


    Note that Visual Studio 2008 has a load test results manager that supports loading, deleting, exporting, and importing load test results.


    Q: How can I turn on logging in the test processes?

    A: There are different processes involved depending on whether you are running your tests locally or against a rig. All have a similar pattern: you need to edit the process's config file to enable logging:


      * Add the following system.diagnostics section to the appropriate config files (change the path to VSTestHost.log if you want):

      <trace autoflush="true" indentsize="4">
          <add name="myListener" type="System.Diagnostics.TextWriterTraceListener" initializeData="c:\VSTestHost.log" />
        <!-- You must use integral values for "value".  Use 0 for off, 1 for 
                 error, 2 for warn, 3 for info, and 4 for verbose. -->
        <add name="EqtTraceLevel" value="3" />

    change the value in this line from “no” to “yes”:

    <add key="CreateTraceListener" value="yes"/> 

    When running locally, make these changes to 
        <Program Files>\Microsoft Visual Studio 8\Common7\IDE\VSTestHost.exe.config

    If running on a rig, change

    <Program Files>\Microsoft Visual Studio 2005 Team Test Load Agent\LoadTest\QTController.exe.config

    <Program Files>\Microsoft Visual Studio 2005 Team Test Load Agent\LoadTest\QTAgentService.exe.config

    <Program Files>\Microsoft Visual Studio 2005 Team Test Load Agent\LoadTest\QTAgent.exe.config


    If more than one of these processes is running on one machine you'll need to give the log files unique names.


    Q: Why is my web test recorder disabled or not appearing?

    See Mike's blog post Diagnosing and fixing Web Test recorder bar issues for a list of known install/configuration problems with the recorder.

    Friday, June 16, 2006 4:58 PM