Server stress testing concepts and methods (TPS/concurrency)

1 Indicators in stress testing

1.1 TPS

TPS is the abbreviation of Transactions Per Second, which refers to the number of transactions processed per second. A transaction is a process in which a client sends a request to a server and the server responds (complete processing, that is, the client initiates the request to get a response). The client starts timing when it sends a request and ends timing after receiving a response from the server. This is used to calculate the time used and the number of transactions completed, and ultimately uses this information to make an evaluation score. A transaction may correspond to multiple requests. Please refer to the transaction operation of the database.

1.2 QPS

QPS is the abbreviation of Queries Per Second, which is the number of queries that can be processed per second (complete processing, that is, from the client initiating a request to receiving a response). It is the number of queries a server can respond to per second, and is a measure of how much traffic a specific query server handles within a specified time.

We can infer from its full English name that it means query. It turns out that on the Internet, the performance of machines serving as domain name system servers is often measured by the query rate per second. Corresponding to fetches/sec, that is, the number of response requests per second. Although nominally it means query, in practice, it is now customary to express the processing capacity of a single interface service in terms of QPS (even if it is not a query operation).

1.3 Average processing time (RT)

RT: Response time, the average processing time required to process a request.

We also generally focus on the average processing time of 90% of requests, because extreme situations may occur due to network conditions.

1.4 Number of concurrent users (concurrency)

The number of users who initiate requests to the interface to be tested per second.

1.5 Conversion Relationship

QPS = number of concurrent connections/average response time

Concurrency = QPS * average response time

For example, 3,000 users (concurrency) access the interface to be tested at the same time. According to statistics from the user side, the average response time for 3,000 users is 1188.538ms. So QPS=3000/1.188538s= 2524.11 q/s.

We can describe this test like this: with 3000 concurrent connections, the QPS is 2524.11 and the average response time is 1188.538ms

1.6 Difference between TPS and QPS

When I started this question, I thought these two should be the same thing, but after seeing their English names on Zhihu, I now think:

QPS refers to the number of queries that can be processed per second, but it is now generally also used to refer to the number of requests that a single service interface can process per second.

TPS is the number of transactions processed per second. If the transaction is completed by only a single service interface, we can also consider it as QPS.

PS: There is also a concept of RPS, request per second. The number of requests per second is similar to QPS and TPS under certain conditions.

2 Stress Testing Methods

We can use stress testing tools to simulate multiple users to stress test the system. There will be an introduction to stress testing tools later.

The test method is to keep a certain total number of requests unchanged, gradually increase the concurrency, and observe the changes in QPS and average response time.

For example, the total number of requests is 10,000, and then the QPS value is tested with a concurrency of 100, and then 200, 300, 400, 500, and so on.

The throughput of a system is usually determined by two factors: TPS and concurrency. Each system has a relative limit for these two values. Under the access pressure of the application scenario, as long as one item reaches the system's maximum value, the system's throughput will not increase. If the pressure continues to increase, the system's throughput will decrease instead. The reason is that the system is overloaded, and other consumption such as context switching and memory leads to a decline in system performance. Here is a stress test chart using the ab tool.

It can be seen from the figure that when the concurrency is 2000, the QPS has reached about 2500, and the subsequent increase in the concurrency remains at 2500, indicating that the QPS of this interface under this configuration is 2500, that is, the system can only process about 2500 requests per second. The subsequent increase in the concurrency will only lead to an increase in the average response time. (PS: Because only 2,500 requests can be processed per second, and there are 7,000 concurrent requests at a time, it will naturally cause request accumulation, resulting in a longer average response time.) We can see that after exceeding 14,000, even the QPS begins to drop sharply, indicating that the system is overloaded and performance begins to drop sharply. In general, we believe that when the average response time reaches a certain value, it is no longer acceptable.

3 Related Documents

Blog on estimating the concurrent volume of IoT devices:

https://www.jb51.net/article/231516.htm

Stress testing tool ab tool:

https://www.jb51.net/article/231502.htm

Node express framework stress test results:

https://www.jb51.net/article/231512.htm

This concludes this article on the concepts and methods of server stress testing (TPS/concurrency). I hope it will be helpful for everyone’s study, and I also hope that everyone will support 123WORDPRESS.COM.