Avg. Response Transaction Time + Think Time does not equal Avg Transaction Time

Question

Hello all,

I have a web performance test. All requests in this test is incapsulated into 1 transaction.

I added this test to the load test and got following results:

Avg. Response Transaction Time = 1.25 sec

Avg. Response Transaction Time = 25.3 sec

Think time profile = ON

Summary Think time for web test = 21 sec

Expected results: Avg. Response Transaction Time + Think time = Avg. Response Transaction Time

Actual results:  1.25 +21 IS NOT equal 25.3 sec. The difference is 3.05 sec.

Could somebody help me to understand it.

Thank you!

Answer 1

Hi YULIYA,

Thank you for posting in the MSDN forum.

Where did you get this result? Did you get it in the Transactions table of Tables?

Maybe you could share us a screen shot about the result.

In addition, if you set the Think time profile=off, and then check the result. As far as I know, if we close the think time, the Ave Response time will be equal to the Ave.Transaction time, see the screen shot provided by Vicky in this thread “Load Testing :: Avg Transaction Time vs Avg Response Time vs Avg Test Time”, hope it could help.

Best Regards,

---

Jack Zhai [MSFT]
MSDN Community Support | Feedback to us

Answer 2

Hello Jack, 

Please, review screenshot bellow. There is difference between Test Time and Transaction time. 

Also I recieve the saim inconsistency regarding Transaction time (39.1sec), Transaction Response  (17.6 sec) and  Think time (21 sec)

17.6+21 = 38.6 is not equal 39.1. 

Thank you,

Yuliya

Answer 3

Hi Yuliya,

Glad to receive your reply.

Could you share us a screen shot like the following screen shot?

For transactions, there are two counters: "Avg. Response Time" and "Avg. Transaction Time". The former is the average of the sum of all of the page times (without the think times), and the latter is the same but includes the think times.

Reference:

Analysing Average Transaction Time in LoadTest Summary

Best Regards,

---

Jack Zhai [MSFT]
MSDN Community Support | Feedback to us

Answer 4

Hi Jack,

Thank you for explanation, but my question is : why AveTransaction Time - Ave Response Time does not equal Think time ?

And also Ave Transation Time  should be equal Avg. Test Time, but it is not.

The difference is huge in case of long test:

Answer 5

Hi Yuliya,

Glad to receive your reply.

As far as I know, the Transaction Time = Response Time + Think time, but we couldn’t control the Avg. time, I try to repro this issue, I get the same result as yours, the Avg. Transaction Time 16.7s, the Avg. Response time 1.75s, the think time is 15s. So I think we couldn't control the Avg.time. But if you turn off the think time, the result will be equal.

In addition, the Avg. Test Time (sec) isn’t related to the Avg. Transaction Time (sec), the test time is the time from starting test to the end of the test, but the Transaction Time = the response time +think time. The response time: From sending request to get the response. Generally, we didn’t compare them.    

Best Regards,

---

Jack Zhai [MSFT]
MSDN Community Support | Feedback to us

Answer 6

Thank you Jack for your explanation.

May I ask you one more related question. 

In your opinion, what category ( Avg. Test Time, Avg Transation Time or Avg. Transation Response Time) should be selected in case of the performance comparison of two tests results.

For example, if comparison would be based on Avg. Test Time the difference might me 60%, but if I select Avg. Transation Response Time, it might be 40%.

I need to mention, that Think time and amount of requests in these two tests are different, but from the user prospective it is One Bussiness Transaction, that is why it can be compared. Think time sould be ON.

I mean, on which counter I can rely on in case of two test results comparison ?

Thank you,

Yuliya

Answer 7

Hi Yuliya,

Glad to receive your reply.

About this new issue, do you mean that you want to report Load Tests Results for Test Comparisons or Trend Analysis? If so, you could check the following article:

http://msdn.microsoft.com/en-us/library/dd728091

In addition, by default, some counters are selected for this report.

Reference:

http://msdn.microsoft.com/en-us/library/dd997707.aspx. Hope it could provide the useful inforamtion.

Best Regards,

---

Jack Zhai [MSFT]
MSDN Community Support | Feedback to us

Answer 8

Thank you , but unfortunately it was not exactly what I was looking for.

This report is available in case of test run comparison for the same test.

I have to different tests ( first for new application version, second test for previos application version). 

For example:

Test A has -  Avg. Test Time = 30 sec

 -  Avg. Transaction Response Time = 10 sec

 - Avg. Transaction Time = 25 sec

  - Amount of test that was finished (during 10 min) = 50

Test B has -  Avg. Test Time = 40 sec

 -  Avg. Transaction Response Time = 12 sec

 - Avg. Transaction Time = 32 sec

- Amount of test that was finished (during 10 min) = 45

If compare results would be based on the Finished Test value, the difference would be 10%.  Test A would be faster than Test B for 10%. (50-45)/50 = 0.1

If compare results would be based on the Avg. Transaction Time , the difference would be  (25-32)/25 = 28$.  Test A would be  faster than Test B for 28%

The question is which counter I should rely on ?

Answer 9

Hi Yuliya,

Sorry for my delay.

We know that Transaction Time = the response time +think time. The response time: From sending request to get the response. The test time:  The average time to execute the test during the load test run. As my understanding, it will include the time to handle the code before sending the request.

A: If compare results would be based on the Finished Test value, the difference would be 10%. Test A would be faster than Test B for 10%. (50-45)/50 = 0.1

B: If compare results would be based on the Avg. Transaction Time, the difference would be (25-32)/25 = 28$. Test A would be faster than Test B for 28%

I’m not very sure why you want to compare the different tests with the above results, but if you run them for different times, I think you will get the different results.

A:  we just could make sure that the test A is faster than test B. Maybe the test A has more code to handle or others.

B: The same think time, maybe the response sent back from the server A is faster than the server B or the test A sent few requests or others.

Based on my description, maybe it is not very clearly. But I just mean that we couldn’t make sure that the percentage is the same, I'm afraid that it has the same result even though we compared the same test with different results.

So about this issue “which counter I should rely on?”, it relies on what results we want to compare.

Best Regards,

---

Jack Zhai [MSFT]
MSDN Community Support | Feedback to us

Answer 10

Thak you for answer.

Let me know if I am wrong, but based on your explanation I can assume that the rule Avg.Transaction Time = Avg. Response Time + Think time would work only in case Think time option OFF.

"As far as I know, the Transaction Time = Response Time + Think time, but we couldn’t control the Avg. time, I try to repro this issue, I get the same result as yours, the Avg. Transaction Time 16.7s, the Avg. Response time 1.75s, the think time is 15s. So I think we couldn't control the Avg.time. But if you turn off the think time, the result will be equal."

Answer 11

I agree with you! J

Best Regards,

---

Jack Zhai [MSFT]
MSDN Community Support | Feedback to us