Possible bug in calculation of TPS for a test

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Possible bug in calculation of TPS for a test

colinmain
Hi,
I have been investigating an issue reported by one of my users and it appears to be the result of a minor bug.

The system under test usually runs at about 300 TPS during a 1 hour test but for about 45 individual seconds scattered throughout that test the system under test does not respond at all for some seconds (which is the subject of a separate investigation). What is recorded in the results CSV file is along the lines of the following:

   Time      TPS   Mean Test Time (ms)
 . . . . .       . .            . . . . .
15:10:22    270            14.27
15:10:23    303            12.34
15:10:26    880            13.09
 . . . . .       . .            . . . . .

When the average TPS for the whole test is calculated it appears to be taken as the average of all the TPSs within the file rather than the total transactions divided by the total test time. In the case of the above, nGrinder would report 1453 / 3 = 484 TPS, but the true value should be 1453 / 5 = 291 TPS.

For most tests this is not an issue as the differences tend to be very small (~1%) but for shorter ones, or for systems giving a very low throughput, this could be sigificant.

(We are using version 3.2.3.)

Please could you check this out? - I would hope it is a relatively simple thing to check

Many thanks,

Colin Main
Colin