Transaction per second (TPS) is the number of transactions executed per second. In other words, it can be calculated based on how many transactions are executed over a certain duration of the test and then calculate it for a second.
For example, if a vuser executes 6 transactions every minute, then the TPS would be 6 transactions/60 sec = 0.10 TPS
There’s an important fact hidden in the above example. It also tells us that if the vuser is able to complete 6 transactions in 60 seconds that means response time for each transaction is 10 sec. i.e.
0.10 transactions –> 1 second
1 transaction –> 10 seconds
Little’s law
Before going into details of the formula derived by little’s law, it is important to understand how response time and pacing are related because these are the key factors that control your TPS if the number of vusers is kept constant.
As we know pacing is inter-iteration gap i.e. it is the pause a vuser takes before executing next iteration. And hence it also helps to control the rate of iterations i.e. rate of transactions i.e. TPS.
Remember, Pacing always overrides Response time when it comes to determining TPS
In the examples below, let’s assume that the vuser’s action() contains only one transaction. In following examples, you may find that the terms ‘transaction’ and ‘iteration’ are used interchangeably. That is just because the action contains only one transaction. Thus one transaction will be fired when one iteration of the action is completed.
Also, to simulate correct TPS, always use the FIXED pacing option available in Run-time settings of Loadrunner. The first two options are less reliable because the response times fluctuate under load and thus affecting the TPS. See the screenshot below.
Example1: Let’s say that we have set Pacing to zero and Think Time to zero.
A vuser performs 5 transactions and response time for each transaction is 10 seconds. Then it will take 5*10 = 50 seconds to complete 5 transactions. i.e. 5/50 = 0.1 TPS. Here, you can see that TPS is controlled by response time.
Example2: Let’s say that we have set pacing to 15 sec and Think Time to zero.
A vuser performs 5 transactions and response time for each transaction is 10 seconds. But the fact that now pacing is greater than response times, it will override the response times and it will control the transaction rate. Thence it will take 5*15 = 75 seconds to complete 5 iterations. i.e. 5/75 = 0.06667 TPS [This is the most correct way of setting up your test scenario to simulate consistent TPS load on the system under test.]
In the second example, as long as the average response times are less than 15 seconds, it will always take 75 seconds to complete 5 iterations i.e. generate 0.06667 TPS
In both the above examples, we also assumed that think time was set to Zero. If, say, it was 2 seconds, total time taken to complete 5 transactions would still be 5*15 = 75 seconds i.e. generate (5/75) = 0.06667 TPS
Once you understand this concept, rest understanding of Little’s law is easy.
If pacing is set to zero, then
Number of vusers = TPS * (Response Time + Think Time)
If pacing is ≠ zero and pacing > (response time + Think Time) , then the above formula would look like this
Number of vusers = TPS * (Pacing)
The fact that TPS is a rate of transactions w.r.t. time, it is also called as throughput.
So Little’s law is
Average number of users in the system = average response time * throughput
N = ( R + Z ) * X
Where, N = Number of users
R = average response time (now you know, it can be pacing too)
Z = Think Time
X = Throughput (i.e. TPS)
Example, If N = 100, R = 2 sec, 100= (2+Z)*X and hence –> If Z=18, X = 5
NOTE: Please do let me know if you see any flaws in the above theory.
[Update on 30-03-2014]: Many readers asked me a question about why I am confusing transactions with iterations in all the examples given in this post. Well, as stated somewhere in the explanation, I have made an assumption that the vuser’s Action() contains only one transaction in it. This may help beginners to understand the concept. So the code in script will look like
vuser_action()
{
lr_start_transaction("LR_01_ViewFullPost");
web_submit_data("ViewPost.aspx",
"Action=http://www.performancengineer.com/ViewPost.aspx?SID={sSID}",
"Method=POST",
"RecContentType=text/html",
"Referer=http://www.performancengineer.com/ViewPost.aspx?SID={sSID}",
"Snapshot=t1.inf",
"Mode=HTTP",
ITEMDATA,
"Name=SID", "Value={aSID}", ENDITEM,
"Name=__VIEWSTATE", "Value={Viewstate_1}", ENDITEM,
"Name=ctl00$DefaultContent$txtDescription", "Value=ok", ENDITEM,
"Name=ctl00$DefaultContent$txtAmount", "Value=1", ENDITEM,
"Name=ctl00$DefaultContent$btnViewPostNow", "Value=Submit", ENDITEM,
"Name=__EVENTVALIDATION", "Value={Eventvalidation_1}", ENDITEM,
LAST);
lr_end_transaction("LR_01_ViewFullPost", LR_AUTO);
lr_think_time(lr_eval_string({pThinkTime}));
}
The Performance Engineer 
very good explanation
LikeLike
Thank you.
LikeLiked by 1 person
Hi
As you have rightly pointed out pacing is inter-iteration gap but in all your example you have used pacing like think time gap between each transaction.
LikeLike
Hi Vijay,
From the way I have explained it appears like think time but it isn’t. Use of think time is explained in later part of second example. Thanks for highlighting it :)
LikeLike
very good explanation of the key concept in a layman’s language, very useful..
LikeLike
Thank you Abhishek. :)
LikeLike
Hey, I think you are confusing Transactions with Iterations.
The gap between Transactions = Think Time
The gap between Iterations = Pacing
And don’t you think that, in example n°2, Pacing should be add to Transactions Response Time?
LikeLike
Hi Jose,
While writing the examples, I have made an assumption that the vuser_action() contains only one transaction. So in turn, each iteration will simulate only one transaction :) That’s why it may sound confusing at first but it’s not. To make it crystal clear, I have updated my post with example code (see last section of the post). Thanks for pointing it out.
And the answer to your second question is NO, pacing shouldn’t be added to transaction response time. If pacing > response time, pacing always overrides Response time when it comes to determining TPS. If pacing < response time, the response time determines TPS. In the later case, if there are too many fluctuations in response times, so it will be in TPS; thereby simulating inconsistent load on the system under test. This may give incorrect results of how system performs under certain TPS load. So it is always advised that you should set pacing at least 2x/3x times your average response time. Hope it is clear.
LikeLike
Hello Dinesh,
How the concurrent user volume will be considered into the TPS, since each user can be associated to a thread for the transaction where in the equation would you consider these.
Thanks
LikeLike
Usually, the business requirements would state how much load the site is expected to have. Number of concurrent users and TPS go side by side. So it is important to consider both while setting up load test scenario.
For example, by adjusting pacing and think time,
one can run a load test with 1000 concurrent users but simulating 10 TPS.
OR one can run a load test with just 10 concurrent users and simulating 10 TPS.
What differs here is that former test would create 1000 threads on the load generator(s), which would create 1000 concurrent sessions on the application.
Later test would create only 10 threads and 10 concurrent sessions on the application.
For a static website (with no logins), the number of concurrent users may not even matter. So using just 10 concurrent users to simulate 10 TPS load would suffice.
Whereas, in case netbanking website where user sessions are important, running the same 10 TPS load with 1000 concurrent users would make more sense.
LikeLike
Hi Dinesh,
Could you please explain how each transaction will take 15 sec in your second example
As per my understanding, if pacing is set to 15 secs then each transaction will take Res time + 15 secs which will be more than 15 secs.
LikeLike
Hi Santiago,
Your understanding is also correct in a specific scenario. As you know, Loadrunner offers three options on when to start a new iteration.
1. Start new iteration as soon as previous iteration ends – This is the default option and least used because the control over the projected load is simply lost.
2. Start new iteration After previous iteration ends; with fixed/random delay of xx sec – Represents a much more real-life like load but as TPS would still depend on response time of transactions. So in a way, you still loose control on the load.
3. Start new iteration At fixed/random intervals, every xx sec – Best and most controlled option of the three. Loadrunner will try to start the next iteration after a fixed space, thereby giving you more control on the load during the test.
In my examples in the post, I have used third option. You are talking about option two.
LikeLike
Can anyone tell how think time must be selected and on what basis.
LikeLike
\i had one question in interview i.e Question: if there are 500 requests sent and the avg response time and TT is 10sec then how many Vusers are used?
Please provide me answer for the above question with detail explaination.
LikeLike
Hi Sreekanth,
If you want to calculate Vusers, you would need the duration for which those 500 requests were sent.Was there any mention of that?
If I assume zero pacing and zero think time,
Even 1 Vuser can fire 500 requests back to back thereby taking total of 5000 seconds to finish.
With 10 Vusers, each user can fire 50 requests and take 500 seconds to finish.
with 100 Vusers, each user would fire 5 requests and take only 50 seconds to finish :)
LikeLike
Thanks for the post and it is well explained in detail. :)
LikeLike
Glad to hear that :)
LikeLike
using little’s formula once we get the No. of Vusers per second, then how can we can calculate No. of Vusers per minutes ? just multiplying by 60 ? please correct me if i am wrong.
LikeLike
Hi Maruti,
Did you mean by “No. of transactions per second”? If yes, then RIGHT, you can multiply by 60 to get number of transactions per second :)
LikeLike
The explanation is good, though i have observed in two ways it should be corrected.
1.in one line you are saying transaction and iterations are same, which is not correct–May be typo mistake
2.In the second example, 1st instance you are taking think time=0 then TPS=0.06667 TPS, and in the 2nd instance you are taking think time=2 then also TPS=0.06667, which is not correct. It is clear that when think time increases number of requests will decreased, ultimately response time will also decreased. When think time decreases number of request will increased.
LikeLike
Hi Pavan,
Thank you for your feedback.
1. In the examples I assumed that action() contains only one transaction. Thus one transaction will be fired when one iteration of the action is completed. That’s why you see that the terms ‘transaction’ and ‘iteration’ are used interchangeably. That is just because the action contains only one transaction. I have now added a small explanation just above example 1. Also added a bit of explanation at the end of post.
2. It is not necessary that when think time increases, the throughput will decrease. It depends on how much pacing you have set. In the example, think time is set to 2sec. So as long as [response time + think time] remains less than 15sec [pacing], the throughput will remain consistent. If think time was set to say 12sec then it [10sec+12sec] would have caused drop in throughput because it would exceed the fixed pacing of 15 seconds. Hope that clarifies.
LikeLike
Could you please tell, if (Response Time + Think Time) = Pacing, then which example we need to chose.
Also can you please explain how to calculate TPS if there are 1 iteration contains 5 transactions.
LikeLike
Hi Ruchi,
If (Response time + think Time) = Pacing, then you should choose the second part of example 2 and use any of the two formulas given below that. It should work.
If 1 iteration contains 5 transactions then you should ensure that pacing is greater than or equal to the combined response times of all 5 transactions in order to achieve consistent TPS. Suppose the response times are t1=2sec, t1=5sec, t1=3sec, t1=6sec, t1=4sec; then give pacing equal to or greater than 20sec and use the formula (Number of vusers = pacing * TPS) to calculate TPS. Hope that clarifies. Thanks for your comments.
LikeLike
Hi,
Instead of specifying think time and response time separately can I use the elapsed time as an additive which I get after running an iteration in vugen.
(T+R) = Elapsed time…
Pls correct me if am assuming something wrong and let me know how to know the exact response time from an iteration.
LikeLiked by 1 person
Hi RajPradeep,
You can use Elapsed time instead of (T+R) but keep following in mind…
1. Depending on how many lr_think_time(x) you have put in your action() for various transactions and which runtime settings you use for think time, elapsed time will increase for the script replay.
2. Ensure you replay script for at least 10 iterations to see whether elapsed time remains consistent. If its consistent, use that as the base response time to calculate TPS/Vusers. If its not consistent, probably take average of elapsed time for those 10 iterations and use that.
3. Assuming you want to achieve consistent TPS throughout the load test, set pacing 2x or 3x of the elapsed time (there is no thumb rule, but this usually does the job). This will cater for response time variations (if any) under load which you may not be aware of while measuring the elapsed time.
LikeLike
Reblogged this on sendilsadasivam.
LikeLiked by 1 person
Thank you.
LikeLike
Suppose there is 3 transaction T1, T2, T3 in a single script. Think time is 2sec and response time is 5 sec per transaction and pacing = 20 sec. Then after how much time Iteration 2 is going to start.?
Next question, if TT= 2sec and pacing is 10 sec , so how much Vuser should i use to achieve 2TPS..?
LikeLike
1. The answer to this question lies in the second part of Example 2 in my post. Pacing (20 sec) is greater than [Think Time + response time] (5+2+5+2+5), so next iteration will start 1 sec after T3 is finished. As long as TT+RT is less than Pacing, each new iteration will happen at 20sec interval.
2. If this is related to your first question, you won’t be able to achieve 2 TPS with 10 sec pacing.
N = TPS * Pacing = 2 * 10 = 20 vusers however
N = TPS * (TT+RT) = 2 * (5+2+5+2+5) = 38 vusers.
Theoretically, If the values are correctly set, both the formulas above should return identical results. Meaning, you should set the pacing in such a way that it always remains higher than TT+RT (answer to question 1).
LikeLike
In answer 1,total (TT+RT)=19sec pacing, then next iteration will start after 20 sec..?
can you please check your answer, i got confused ?
LikeLike
See this. https://performancengineer.files.wordpress.com/2013/09/tps-4-thinktime.png
Few examples for your case with FIXED pacing of 20sec and TT of 2sec. Lets say the response times vary between 2 sec to 5 sec.
1. When RT=2: RT+TT=10, next iteration will start after 10 sec
2. When RT=3: RT+TT=13, next iteration will start after 7 sec
3. When RT=4: RT+TT= 16, next iteration will start after 4 sec
4. When RT=5: RT+TT=19, next iteration will start after 1 sec
LikeLike
Hello Sir,
If I want to run the test with 5000 users, with 2hours steady state, 30min ramp-up, 30mins ramp down, then how to calculate the think time and pacing for this load test? Please assume the missing factor if any from your end and explain the calculation.
LikeLike
I want to calculate TPS for a script, below are the script details
Script1 contains 5 transactions, for 5 transaction it takes 22 sec to execute assume i am giving tt 3 sec so total tt in the script will be 4*3=12. how do i calculate tps?
LikeLike
Well explained :)
LikeLike
Thank you 😎
LikeLike
Very good explaination..i would suggest take more examples to explain with different paramters values and calculate.
LikeLike
Sure. There are more examples in comments though. ☺️
LikeLike
Well explained.. very helpful :)
LikeLike
Glad it helped. :)
LikeLike
A vuser performs 5 transactions and response time for each transaction is 10 seconds. Then it will take 5*10 = 50 seconds to complete 5 transactions. i.e. 5/50 = 0.1 TPS. Here, you can see that TPS is controlled by response time.
how you divided by 50. To get transactions per second you have to divide with 60. I am confused here please help me out.
LikeLike
Hi Ram,
Yes, you are confused. What will be the TPS if there were 6 transactions instead? :-)
Let me break it down further for you.
1 transaction in 10 sec (= 0.1 TPS)
2 transactions in 20 sec (= 0.1 TPS)
3 transactions in 30 sec (= 0.1 TPS)
4 transactions in 40 sec (= 0.1 TPS)
5 transactions in 50 sec (= 0.1 TPS)
So, 6 transactions in 60 sec (= 0.1 TPS)
Notice, TPS has remained consistent!
5 transactions completed in 50 sec, not in 60 sec. So you divide by 50 sec.
LikeLike
can u demonstrate Little’s Law with each iteration having 5 transactions.
LikeLike
let’s assume that the response times for the five transactions are T1 = 2sec, T2 = 3sec, T3 = 5sec, T4 = 4sec and T5 = 5sec.
Combined response time of all transactions in one iteration = 2+3+5+4+5 = 19sec.
Let’s say there is think time of 2sec after each transaction. Total think time = 8sec.
Total time taken for 1 iteration would be 19+8 = 27sec. This should be the minimum value for pacing.
We know the response time of the transactions may increase slightly under load, we need to consider that as well. So let us add some buffer to cover that by doubling the pacing value to 54sec.
Thus, N = 54 * X
If you want to inject 2 TPS load, you will need 108 VUsers.
LikeLike
Nice Explanation.
Can you please clarify me the difference between iteration and session.
Is the session contain vuser_init, Vuser_action, Vuser_exit all of three ? or iteration contain only Vuser_Action?
LikeLike
To put it into perspective, one execution of vuser_action is termed as one iteration. If you perform 10 iterations of the script, vuser_action will be executed 10 times whereas vuser_init and vuser_end are executed only once.
A session consists of vuser_init, vuser_action and vuser_end. A session can have multiple iterations.
LikeLike
what if think time is greater than pacing ? Is it possible ?
LikeLike
Try to put that yourself in the little’s law and find out what happens to the TPS :)
LikeLike
what is the relation between transactions per second and requests per seconds in terms of an architecture containing load balancer, application and database.
LikeLike
Transactions per second (TPS) and requests per second (RPS) are very vague terms on its own and have different interpretations depending on what you are looking at in the technology stack of the application and depending on the business. Also, at times these terms are used interchangeably. For example, consider a flight booking website,
TPS from a business user perspective would be the number of tickets booked per hour.
From the database perspective, it could be the number of queries executed during that hour.
From an application and load balancer perspective, it would be the number of HTTP requests involved in the flow.
So on and so forth.
LikeLike
Hello. Thanks for this post. In order to properly calculate the value we need from Little’s Law, what specific information would the business/customer need to provide us? Can you list these datapoints out for me? Thank you.
LikeLike
Little’s law is an equation, right? So you would need values of n-1 variables present in the equation so that you can derive the value of the nth variable. For example, if you want to identify the number of users to run the load test, you would need R, Z and X. While this looks straightforward, the business may not provide these values directly. Instead, for example, you may be given a transaction/session log from which you will need to determine the session length (helps to derive think time and pacing), average response times, number of transactions over a period of time (helps to derive TPS) and so on.
LikeLiked by 1 person
I really appreciate that you took the time to reply to me despite this article being years old. Really cool of you, and thank you for that.
So… yes, it’s an equation. I suppose my question is, as a novice to performance testing dealing with customers that do not have a mature process, what would be the layman’s version of questions to ask business analysts, devs, etc. to get the missing values for Little’s Law? I’m basically looking to put together a checklist and would love to just outright ask questions to obtain N, R, Z, and/or X values.
In my experience, most businesses can only provide N, because they tend to know what their peak user count is. Often they won’t even know that.
LikeLike
Asking the right people the right questions is the key :) The second part of my previous answer has some hints on what information you can use to derive those values. For example, I can look into Google Analytics for a given average day, during a peak hour and get stats like this:
1. 4,000 visitors in 60 minutes
2. 20,000 page views
3. avg page views 5
4. Avg time on site 7 minutes
Then you can use this info to derive the values for little’s law.
4000 users in 1 hour (60 minutes), 7 min time on site
i.e. 60 minutes / 7 min = 8.5
4000 / 8.5 = ~470 Users
LikeLiked by 1 person
So, if a customer had access to something like Google Analytics, or other logs, I could approach them and theoretically get values for N, R, Z, and/or X values rather “easily”?
Let’s say I want to perform a load test, but the customer hasn’t given me much data. It’s clear that I could outright ask a customer for “N”, with a question like “What is the number of visitors you had on the system at the peak hour of the peak day?”.
For R, I could ask them “What was the average response time according to the logs?” Would such a question sense to a competent dev/BA?
For X, I could ask them “What were the average transactions per second during this period?” Would such a question sense to a competent dev/BA?
Now, given that I’ve been provided (hopefully), N, R, and X, I can use the formula above (N = ( R + Z ) * X) to calculate the value of Z (think time).
With all 4 values, I can design the workload characteristics of my script in Loadrunner’s runtime settings to have a valuable load test?
Using real numbers, in this hypothetical, if the business tells me N is 5000, R is 3, and Z is 3, then that means X = 833.34 TPS. That means that if I run a script with 5000 virtual users with a think time of 3 seconds, a pacing value of 3 seconds, then my test should achieve 833 transactions per second? If the application hits far less TPS than that expected value, does that mean there’s an issue with the application or is the load test badly designed, etc.?
LikeLike
All these questions would make sense to the Devs if not BAs.
Worst case, be ready for the stats to be thrown at you so you can derive those yourself. This would be an opportunity to understand not just the averages but also find out 90th, 95th percentiles response time patterns, session length etc.
Your calculations in the last paragraph are incorrect. Read example 2 carefully and notice the RED caution note that starts with “Remember…”.
Good Luck and thanks for Stopping by.
LikeLike
This formula should also have a component called Number of Transactions
Generally we have more than one transaction in script , say we execute a script containing “No of Transaction” transactions with single user and have a response time of RT seconds. Considering TT as think time and Pac as pacing , then the formula should look like
No of Users=(TPS*(Resp Time RT+(Think time TT*No of Transactions) +Pacing pac))/(Number of Transaction in Script)
LikeLike
You are correct about having a formula with the Number of Transactions as a component. However, the formula provided by you is slightly incorrect.
You can simplify it by using this formula: Number of Vusers = TPS * (Pacing)
Where
Pacing >= No of transactions x (Avg RT of each transaction + TT after each transaction)
OR
Pacing >= No of transactions x (RT1+TT1+RT2+TT2+RT3+TT3+….+RTn+TTn)
LikeLike
as per my understanding pace time is the time delay we put between each iteration.
as per your example eac transaction takes 2 seconds and if there is 5 total times spent is 10 seconds without think time and pace time.
when it comes to pace time , there is a delay of 15 seconds in each of the iteration. so if user do 5 iterations
15*5 is the pace time and 5(transaction)*2(each taking 2 seconds)*5(no of iterations) is the actual response time which is 50 seconds.
so total time taken by user for the scenario for 5 iteration is 50+75 and not 75 alone
hope i am making sense. please correct me if i am wrong.
LikeLike
Hi Sumit,
Thanks for stopping by.
Loadrunner provides different options to simulate pacing. Your understanding is absolutely correct if you use the option “After the previous iteration ends”. https://performancengineer.files.wordpress.com/2013/09/run-time-settings.png. However, this option doesn’t guarantee consistent TPS during the test because the response times of transactions may vary under load. Your calculations remain valid as long as the response time remains 2 sec for each transaction :)
If you want to simulate consistent TPS, use the last option of FIXED pacing (I referred this one in all the examples provided in the post). So taking your example,
Pacing =15 sec,
Total of response time of each iteration = 5(transaction)*2(each taking 2 seconds) = 10 sec
Total time taken by user for the scenario for 5 iterations will be 15 (pacing) * 5 (iterations) = 75 sec.
So where did the 10 sec (response time of each iteration) go?? It gets encapsulated by the pacing. So say, the first iteration takes 10 sec, loadrunner waits for another 5 sec, so 15 sec pacing is reached and then starts the next iteration. If the second iteration takes 12 seconds (due to slower response from the server), Loadrunner waits for 3 sec to reach 15 sec pacing and then starts 3rd iteration. Hope that helps. Test it yourself :)
LikeLike