"User Load" not decrementing as users finish.

Question

Hi,

Model:
1 Webtest in a Load Test.
Load Profile: 50 users Stepped.
Run Config: Iterations= True, 50 Iterations.

With the above config, the load test happiliy steps the 50 users in a controlled manner and the runtime graph plots a nice upward trend to 50 users during the run.
However, as the iterations start to complete the 'Userload' graph and value remains at 50 users. 
Why would this be the case?

Is it related to the outstanding iterations problem i have in the thread below?
http://social.msdn.microsoft.com/Forums/en-US/vstswebtest/thread/11ca3cf9-e5a6-4731-b44e-5f33ca1ddfca


All i'm doing is running 50 users against one webtest for 50 iterations. So i was expecting a graph that "ramped up, plateaus, then ramps down" like a hump back.
I am putting my AUT under that exact load pattern (i can verify it in the AUT databaseS) but VSTS is not reporting it correctly.

Thanks for you help.
Richard.

Answer 1

Richard,

What is your step duration and about how long does each test take to execute?  Are you sure you even get to 50 users?  The test iterations will limit the total number of tests you run to 50 but I don't think that is what you want.   Say you have a step load pattern starting at 1  user and increasing by 1 user every 10 seconds.  Also assume your test takes 10 seconds (not likely)  Here is what your test would look like in tabular form:

Time   Users  Tests Total Tests
0       1         1       1
10     2         2       3
20     3         3       6
30     4         4       10
40     5         5       15
50     6         6       21
60     7         7       28
70     8         8       36
80     9         9       45
90     5         5       50


The test iterations will stop your test from ever hitting the 50 users.  Your test is running up to some number of concurrent users (dependant upon how long each test takes including think time) then just stopping because the 50 test threshold is hit.


Could you tell us what you are trying to achieve with your test (capacity, expected response time at a certain load, just trying to figure out the test tool...)  and maybe we can suggest a better configuration.

---

Tim

Answer 2

Hi Tim,

I am trying to replicate a Test Model that i currently have in our Rational Robot/TM Performance Toolset.

As mentioned in the other thread, i don't want the test duration to be governed by time. It just needs to stop when the test has completed.
All i'm doing is running a 50 user load test all running the same webtest concurrently. The webtest has a loop in it (*150) and i want it to loo through it 150 times which it does.

So I have a test mix with one webtest in it, That webtest has a loop in it (*150).
As i don't want the test to be time dependant, i have to set the Run Config: Iterations = True.
If a run this with 50 users, only one user is run as i have discovered that the iterations value of the Run Config is a global iterations value for all virtual users, not for just one. Therefore i have to set the iterations value to 50 to match the number of virtual users in the test.
My load profile works correctly and is set to 50 users, stepping in 1 user every 3 seconds. So within 2.5 mins, all 50 users are started.

With these setting, the test correctly starts the 50 users over the duration i specified and the each of the 50 users runs the webtest once and they all gracefully logout of the AUT and stop. (All good)

But as mentioned, i'm expecting the userload value to decrement as the the webtest in the test mix finish executing but it doesn't.

Thanks
Richard.






  

Answer 3

I see, each test takes considerably more than 10 seconds - probably more than 150 seconds or you wouldn't reach the 50 concurrent users.  I did a quick test and saw the same thing you saw.  I'll have to think about it some more because I'm not sure what I think it should be but MS would probably have to explain why.  If you look at the Tests Running counter you will see what you want.

---

Tim