Loadrunner: Understanding Transaction Per Second (TPS)

Transaction per second (TPS) is the number of transactions executed per second. In other words, it can be calculated based on how many transactions are executed over a certain duration of the test and then calculate it for a second.

For example, if a vuser executes 6 transactions every minute, then the TPS would be 6 transactions/60 sec = 0.10 TPS

There’s an important fact hidden in the above example. It also tells us that if the vuser is able to complete 6 transactions in 60 seconds that means response time for each transaction is 10 sec. i.e.

0.10 transactions –> 1 second

1 transaction –> 10 seconds

Little’s law

Before going into details of the formula derived by little’s law, it is important to understand how response time and pacing are related because these are the key factors that control your TPS if the number of vusers are kept constant.

As we know pacing is inter-iteration gap i.e. it is the pause a vuser takes before executing next iteration. And hence it also helps to control the rate of iterations i.e. rate of transactions i.e. TPS.

Remember, Pacing always overrides Response time when it comes to determining TPS

In the examples below, let’s assume that the vuser’s action() contains only one transaction. In following examples, you may find that the terms ‘transaction’ and ‘iteration’ are used interchangeably. That is just because the action contains only one transaction. Thus one transaction will be fired when one iteration of the action is completed.

Example1:  Let’s say that we have set Pacing to zero and Think Time to zero.

A vuser performs 5 transactions and response time for each transaction is 10 seconds. Then it will take 5*10 = 50 seconds to complete 5 transactions. i.e. 5/50 = 0.1 TPS. Here, you can see that TPS is controlled by response time.

Example 1

Example2:  Let’s say that we have set pacing to 15 sec and Think Time to zero.

A vuser performs 5 transactions and response time for each transaction is 10 seconds. But the fact that now pacing is greater than response times, it will override the response times and it will control the transaction rate.  Thence it will take 5*15 = 75 seconds to complete 5 iterations.  i.e. 5/75 = 0.06667 TPS [This is the most correct way of setting up your test scenario to simulate consistent TPS load on the system under test.]

Example 2

In the second example, as long as the average response times are less than 15 seconds, it will always take 75 seconds to complete 5 iterations i.e. generate 0.06667 TPS

In both the above examples, we also assumed that think time was set to Zero. If, say, it was 2 seconds, total time taken to complete 5 transactions would still be 5*15 = 75 seconds i.e. generate (5/75) = 0.06667 TPS

Response time plus Think time

Once you understand this concept, rest understanding of Little’s law is easy.

If pacing is set to zero, then

Number of vusers = TPS * (Response Time + Think Time)

If pacing is ≠ zero and pacing > (response time + Think Time) , then the above formula would look like this

Number of vusers = TPS * (Pacing)

The fact that TPS is a rate of transactions w.r.t. time, it is also called as throughput.

So Little’s law is

Average number of users in the system = average response time * throughput

N  =  ( R + Z )  *  X

Where, N = Number of users

R = average response time (now you know, it can be pacing too)

Z = Think Time

X = Throughput (i.e. TPS)

Example, If N = 100, R = 2 sec, 100= (2+Z)*X and hence –> If Z=18, X = 5

NOTE: Please do let me know if you see any flaws in the above theory.

[Update on 30-03-2014]:  Many readers asked me a question about why I am confusing transactions with iterations in all the examples given in this post. Well, as stated somewhere in the explanation, I have made an assumption that the vuser’s Action() contains only one transaction in it. This may help beginners to understand the concept. So the code in script will look like

vuser_action()
{
  lr_start_transaction("LR_01_ViewFullPost");
  web_submit_data("ViewPost.aspx",
    "Action=http://www.performancengineer.com/ViewPost.aspx?SID={sSID}",
    "Method=POST",
    "RecContentType=text/html",
    "Referer=http://www.performancengineer.com/ViewPost.aspx?SID={sSID}",
    "Snapshot=t1.inf",
    "Mode=HTTP",
    ITEMDATA,
    "Name=SID", "Value={aSID}", ENDITEM,
    "Name=__VIEWSTATE", "Value={Viewstate_1}", ENDITEM,
    "Name=ctl00$DefaultContent$txtDescription", "Value=ok", ENDITEM,
    "Name=ctl00$DefaultContent$txtAmount", "Value=1", ENDITEM,
    "Name=ctl00$DefaultContent$btnViewPostNow", "Value=Submit", ENDITEM,
    "Name=__EVENTVALIDATION", "Value={Eventvalidation_1}", ENDITEM,
    LAST);
  lr_end_transaction("LR_01_ViewFullPost", LR_AUTO);
  lr_think_time(lr_eval_string({pThinkTime}));
}

 

Advertisements

43 thoughts on “Loadrunner: Understanding Transaction Per Second (TPS)

  1. Hi
    As you have rightly pointed out pacing is inter-iteration gap but in all your example you have used pacing like think time gap between each transaction.

    Like

  2. Hey, I think you are confusing Transactions with Iterations.
    The gap between Transactions = Think Time
    The gap between Iterations = Pacing

    And don’t you think that, in example n°2, Pacing should be add to Transactions Response Time?

    Like

    • Hi Jose,
      While writing the examples, I have made an assumption that the vuser_action() contains only one transaction. So in turn, each iteration will simulate only one transaction :) That’s why it may sound confusing at first but it’s not. To make it crystal clear, I have updated my post with example code (see last section of the post). Thanks for pointing it out.
      And the answer to your second question is NO, pacing shouldn’t be added to transaction response time. If pacing > response time, pacing always overrides Response time when it comes to determining TPS. If pacing < response time, the response time determines TPS. In the later case, if there are too many fluctuations in response times, so it will be in TPS; thereby simulating inconsistent load on the system under test. This may give incorrect results of how system performs under certain TPS load. So it is always advised that you should set pacing at least 2x/3x times your average response time. Hope it is clear.

      Like

      • Hello Dinesh,

        How the concurrent user volume will be considered into the TPS, since each user can be associated to a thread for the transaction where in the equation would you consider these.

        Thanks

        Like

      • Usually, the business requirements would state how much load the site is expected to have. Number of concurrent users and TPS go side by side. So it is important to consider both while setting up load test scenario.
        For example, by adjusting pacing and think time,
        one can run a load test with 1000 concurrent users but simulating 10 TPS.
        OR one can run a load test with just 10 concurrent users and simulating 10 TPS.

        What differs here is that former test would create 1000 threads on the load generator(s), which would create 1000 concurrent sessions on the application.
        Later test would create only 10 threads and 10 concurrent sessions on the application.

        For a static website (with no logins), the number of concurrent users may not even matter. So using just 10 concurrent users to simulate 10 TPS load would suffice.
        Whereas, in case netbanking website where user sessions are important, running the same 10 TPS load with 1000 concurrent users would make more sense.

        Like

      • Hi Dinesh,
        Could you please explain how each transaction will take 15 sec in your second example

        As per my understanding, if pacing is set to 15 secs then each transaction will take Res time + 15 secs which will be more than 15 secs.

        Like

      • Hi Santiago,

        Your understanding is also correct in a specific scenario. As you know, Loadrunner offers three options on when to start a new iteration.
        1. Start new iteration as soon as previous iteration ends – This is the default option and least used because the control over the projected load is simply lost.
        2. Start new iteration After previous iteration ends; with fixed/random delay of xx sec – Represents a much more real-life like load but as TPS would still depend on response time of transactions. So in a way, you still loose control on the load.
        3. Start new iteration At fixed/random intervals, every xx sec – Best and most controlled option of the three. Loadrunner will try to start the next iteration after a fixed space, thereby giving you more control on the load during the test.

        In my examples in the post, I have used third option. You are talking about option two.

        Like

  3. \i had one question in interview i.e Question: if there are 500 requests sent and the avg response time and TT is 10sec then how many Vusers are used?

    Please provide me answer for the above question with detail explaination.

    Like

    • Hi Sreekanth,

      If you want to calculate Vusers, you would need the duration for which those 500 requests were sent.Was there any mention of that?
      If I assume zero pacing and zero think time,
      Even 1 Vuser can fire 500 requests back to back thereby taking total of 5000 seconds to finish.
      With 10 Vusers, each user can fire 50 requests and take 500 seconds to finish.
      with 100 Vusers, each user would fire 5 requests and take only 50 seconds to finish :)

      Like

  4. using little’s formula once we get the No. of Vusers per second, then how can we can calculate No. of Vusers per minutes ? just multiplying by 60 ? please correct me if i am wrong.

    Like

  5. The explanation is good, though i have observed in two ways it should be corrected.

    1.in one line you are saying transaction and iterations are same, which is not correct–May be typo mistake
    2.In the second example, 1st instance you are taking think time=0 then TPS=0.06667 TPS, and in the 2nd instance you are taking think time=2 then also TPS=0.06667, which is not correct. It is clear that when think time increases number of requests will decreased, ultimately response time will also decreased. When think time decreases number of request will increased.

    Like

    • Hi Pavan,
      Thank you for your feedback.
      1. In the examples I assumed that action() contains only one transaction. Thus one transaction will be fired when one iteration of the action is completed. That’s why you see that the terms ‘transaction’ and ‘iteration’ are used interchangeably. That is just because the action contains only one transaction. I have now added a small explanation just above example 1. Also added a bit of explanation at the end of post.
      2. It is not necessary that when think time increases, the throughput will decrease. It depends on how much pacing you have set. In the example, think time is set to 2sec. So as long as [response time + think time] remains less than 15sec [pacing], the throughput will remain consistent. If think time was set to say 12sec then it [10sec+12sec] would have caused drop in throughput because it would exceed the fixed pacing of 15 seconds. Hope that clarifies.

      Like

  6. Could you please tell, if (Response Time + Think Time) = Pacing, then which example we need to chose.
    Also can you please explain how to calculate TPS if there are 1 iteration contains 5 transactions.

    Like

    • Hi Ruchi,
      If (Response time + think Time) = Pacing, then you should choose the second part of example 2 and use any of the two formulas given below that. It should work.

      If 1 iteration contains 5 transactions then you should ensure that pacing is greater than or equal to the combined response times of all 5 transactions in order to achieve consistent TPS. Suppose the response times are t1=2sec, t1=5sec, t1=3sec, t1=6sec, t1=4sec; then give pacing equal to or greater than 20sec and use the formula (Number of vusers = pacing * TPS) to calculate TPS. Hope that clarifies. Thanks for your comments.

      Like

  7. Hi,
    Instead of specifying think time and response time separately can I use the elapsed time as an additive which I get after running an iteration in vugen.
    (T+R) = Elapsed time…
    Pls correct me if am assuming something wrong and let me know how to know the exact response time from an iteration.

    Like

    • Hi RajPradeep,
      You can use Elapsed time instead of (T+R) but keep following in mind…
      1. Depending on how many lr_think_time(x) you have put in your action() for various transactions and which runtime settings you use for think time, elapsed time will increase for the script replay.
      2. Ensure you replay script for at least 10 iterations to see whether elapsed time remains consistent. If its consistent, use that as the base response time to calculate TPS/Vusers. If its not consistent, probably take average of elapsed time for those 10 iterations and use that.
      3. Assuming you want to achieve consistent TPS throughout the load test, set pacing 2x or 3x of the elapsed time (there is no thumb rule, but this usually does the job). This will cater for response time variations (if any) under load which you may not be aware of while measuring the elapsed time.

      Like

  8. Suppose there is 3 transaction T1, T2, T3 in a single script. Think time is 2sec and response time is 5 sec per transaction and pacing = 20 sec. Then after how much time Iteration 2 is going to start.?

    Next question, if TT= 2sec and pacing is 10 sec , so how much Vuser should i use to achieve 2TPS..?

    Like

    • 1. The answer to this question lies in the second part of Example 2 in my post. Pacing (20 sec) is greater than [Think Time + response time] (5+2+5+2+5), so next iteration will start 1 sec after T3 is finished. As long as TT+RT is less than Pacing, each new iteration will happen at 20sec interval.
      2. If this is related to your first question, you won’t be able to achieve 2 TPS with 10 sec pacing.
      N = TPS * Pacing = 2 * 10 = 20 vusers however
      N = TPS * (TT+RT) = 2 * (5+2+5+2+5) = 38 vusers.
      Theoretically, If the values are correctly set, both the formulas above should return identical results. Meaning, you should set the pacing in such a way that it always remains higher than TT+RT (answer to question 1).

      Like

  9. Hello Sir,
    If I want to run the test with 5000 users, with 2hours steady state, 30min ramp-up, 30mins ramp down, then how to calculate the think time and pacing for this load test? Please assume the missing factor if any from your end and explain the calculation.

    Like

  10. I want to calculate TPS for a script, below are the script details
    Script1 contains 5 transactions, for 5 transaction it takes 22 sec to execute assume i am giving tt 3 sec so total tt in the script will be 4*3=12. how do i calculate tps?

    Like

  11. A vuser performs 5 transactions and response time for each transaction is 10 seconds. Then it will take 5*10 = 50 seconds to complete 5 transactions. i.e. 5/50 = 0.1 TPS. Here, you can see that TPS is controlled by response time.

    how you divided by 50. To get transactions per second you have to divide with 60. I am confused here please help me out.

    Like

    • Hi Ram,
      Yes, you are confused. What will be the TPS if there were 6 transactions instead? :-)

      Let me break it down further for you.
      1 transaction in 10 sec (= 0.1 TPS)
      2 transactions in 20 sec (= 0.1 TPS)
      3 transactions in 30 sec (= 0.1 TPS)
      4 transactions in 40 sec (= 0.1 TPS)
      5 transactions in 50 sec (= 0.1 TPS)
      So, 6 transactions in 60 sec (= 0.1 TPS)

      Notice, TPS has remained consistent!
      5 transactions completed in 50 sec, not in 60 sec. So you divide by 50 sec.

      Like

    • let’s assume that the response times for the five transactions are T1 = 2sec, T2 = 3sec, T3 = 5sec, T4 = 4sec and T5 = 5sec.
      Combined response time of all transactions in one iteration = 2+3+5+4+5 = 19sec.
      Let’s say there is think time of 2sec after each transaction. Total think time = 8sec.
      Total time taken for 1 iteration would be 19+8 = 27sec. This should be the minimum value for pacing.
      We know the response time of the transactions may increase slightly under load, we need to consider that as well. So let us add some buffer to cover that by doubling the pacing value to 54sec.

      Thus, N = 54 * X
      If you want to inject 2 TPS load, you will need 108 VUsers.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s