Use ab tool to perform API stress test on the server

1 A brief introduction to system throughput

The throughput (pressure-bearing capacity) of a system is closely related to the CPU consumption, external interfaces, IO, etc. of the request.

The higher the CPU consumption of a single request, the slower the external system interface and IO impact speed, and the lower the system throughput, and vice versa.

Several important parameters of system throughput: TPS, concurrency, response time

• TPS: The number of transactions processed per second
• Concurrency: The number of requests (transactions) processed by the system at the same time
• Response time: Generally take the average response time

TPS=concurrency/average response time

Here, if the transaction is a single interface request, we can also consider TPS to be QPS.

Here are some examples:

For example, 3,000 users (concurrency) access the interface to be tested at the same time. According to statistics from the user side, the average response time for 3,000 users is 1188.538ms. So TPS=3000/1.188538s= 2524.11 q/s.

We can describe this test like this: with 3000 concurrent connections, the TPS is 2524.11 and the average response time is 1188.538ms

Tps: In actual testing, it is shown as follows:

The throughput of a system is usually determined by two factors: TPS and concurrency. Each system has a relative limit for these two values. Under the access pressure of the application scenario, as long as one item reaches the system's maximum value, the system's throughput will not increase. If the pressure continues to increase, the system's throughput will decrease instead. The reason is that the system is overloaded, and other consumption such as context switching and memory leads to a decline in system performance.

The actual performance is that TPS first increases and then decreases. We need to find the performance inflection point. And get the limiting bottleneck.

2 Test Methods

References (detailed output description):

https://www.jb51.net/article/231492.htm

2.1 Client Testing Tools

We use the apacheBench tool for testing.

Ubuntu installs ab:

sudo apt-get install apache2-utils 

The default login port in Linux can only open 1024 files. Because everything in Linux is a file, the number of ab concurrency is limited by the number of open files. It needs to be modified using ulimit -n 10000 (number of open files) to support larger concurrency. I tested and modified it to 15000.

2.1.1 GET method

ab -n 100 -c 100 https://www.baidu.com/index.html

-n: total number of requests

-c: Number of concurrent users.

-url: API to be tested.

When the number of test requests is small, the test is completed quickly and no intermediate process is displayed. When the number of requests is large, the current number of completed requests will be displayed in separate lines.

2.1.2 POST method

ab -n 10 -c 1 -T 'application/x-www-form-urlencoded' -H "Authorization:Bearer 2393d8db9b9d7f4b9d1570cc8776bca69b421b62" -p ./post http://172.28.28.17:3017/oauth2/token

• -H: You can set the response header
• -T: Post http header type defaults to text/plain
• -P: Post body content, ab requires writing it in a file, -p is followed by the file directory, the file content is such as name=hello&password=1234

2.1.3 Test Results Interpretation

Here is the test output of ab:

ab -n 10 -c 2 The above figure shows the result of total request 10 and concurrency 2.

The output information we are mainly concerned with is:

• Concurrency Level: 10 //Concurrency level, that is, the number of concurrent requests, the number specified by the -c parameter in the request
• Time taken for tests: 1.093 seconds //Total time taken for this test
• Complete requests: 100 //The total number of requests initiated in this test
• Failed requests: 0 //Number of failed requests. Due to network or server performance reasons, not all initiated requests are successful. By dividing this value by Complete requests, we can calculate the request failure rate, which serves as an important reference for the test results.
• Total transferred: 103314 bytes //The total amount of data transferred refers to the total amount of data received by ab from the server under test, including the text content of index.html and request header information.
• Requests per second: 91.50 [#/sec] (mean) //Average number of requests completed per second: QPS, this is an average value, equal to Complete requests/Time taken for tests=100/1.093=91.50
• Time per request: 109.287 [ms] (mean) //From the user's perspective, the time required to complete a request (because there are more than one user, the server completes 10 requests, and each user receives a complete response on average, so this value is 10 times the value of the next item.)
• Time per request: 10.929 [ms] (mean, across all concurrent requests) // The time it takes for the server to complete a request.
• Transfer rate: 92.32 [Kbytes/sec] received //Network transfer speed. For large file request tests, this value can easily become a system bottleneck. To determine whether this value is a bottleneck, you need to understand the network conditions between the client and the server being tested, including information such as network bandwidth and network card speed.

Among them, we are most concerned about Requests per second: tps. We regard it as the most important indicator of server performance.

2.2 Server-side detection method

You can use the iftop command and nethogs -d to detect the server network status.

You can monitor server port traffic through the iptables command.

You can use top | grep "node" to judge the memory and CPU.

For cloud testing, you can use the cloud host backend, but the background parameters are average values ​​after minutes.

It seems like this is a stupid way to measure merit.

3 Actual Test

Using apacheBench, you can write shell scripts to perform multiple tests. You can put the API to be tested into the API array and modify the number of loops to test multiple APIs at a time and generate a key parameter xls file. Now it seems that I was too naive to have such an idea.

3.1 Shell Script

Instructions for use: a is the total number of requests, b is the number of concurrent users, i.e. a[0] corresponds to b[0]. The first parameter passed in is the address of the API server to be tested, and the second is the parameters required by the API. The api is set in the api array. To add multiple APIs or agree to multiple sets of API tests, please modify the loop count.

echo "you maybe use this sh like:"$0" serverIP userParam"
a=(1000 2000 4000 6000 7000)#Total number of requests to be tested b=(50 100 200 300 400)#Number of concurrent users runTime=$(date +%Y%m%d%H%M%S)

if [ -z "$1"]
then
	serverip="http://127.0.0.1"
else
	serverip=$1
fi

if [ -z "$2"]
then
	param="deviceid=XXX&bindingplatform=XXX&bindingid=XXX"
else
	param=$2
fi

filename=${runTime}"-test.log"
touch filename

#api=('XXX'${param} 'XXX'${param} '/users/account')
api=('XXX'${param})
echo "********webserver test info****************"
echo "testTime :"$(date) 
echo "LogName :"${filename}
echo "serverIP :"${serverip}
echo "userparam:"${param}
echo "********webserver test info****************" 
#echo ${filename}

for j in {0..0}#The number of APIs to be tested is the number of API arrays do
	echo "API test:"${serverip}${api[j]}
	for i in {0..4}#The number of API tests to be tested is 5 times, which means that the corresponding ab array has five values ​​do
		ab -r -k -n ${a[i]} -c ${b[i]} -C ${param} ${serverip}${api[j]} | grep -e"Document Path:" -e "Complete requests:" -e "Concurrency Level:" -e"Failed requests:" -e"Time taken for tests:" -e "Requests per second:" -e "Time per request" -e"Total transferred: " >> ${filename}
	done
done
sed -i 's/^.\{24\}//g' ${filename}# Generate a txt file according to the time and extract it according to the above parameters.
export LD_LIBRARY_PATH=
./change ${filename} ${runTime}"report.xls"#chang The function is to convert the key data in txt into xls file.
rm ${filename} 

3.2 C++ Extractor: Using libxl.h

#include <iostream>
#include <fstream>
#include <string>
#include "libxl.h"
using namespace std;
using namespace libxl;
int main(int agrc, char *argc[])
{

	//cout << "helloworld" << endl;
	fstream f;
	ifstream ifile(argc[1]);
	string temp;
	int i = 0, j = 1, k = 0;

	Book* book = xlCreateBook();//Create an instance of XLS (Execl97-03) in binary format. You must call this function to create an object to operate Excel before using it. //book->setKey(......);//If you have purchased the library, set the corresponding key. If you have not purchased it, do not use this line if (book)//Whether the instance was created successfully {

		Sheet* sheet = book->addSheet("Sheet1"); //Add a worksheet		
		for(i=0;i<30;i++)
		{
			for(j=0;j<10;j++){
				sheet->setCol(i, j, 20); //Set column width, format, etc. }
		}
		i=0;
		j=1;
		
		if (sheet)
		{
			sheet->writeStr(j, 0, "API");
			sheet->writeStr(j, 1, "Concurrency Level");
			sheet->writeStr(j, 2, "Time taken for tests");
			sheet->writeStr(j, 3, "Complete requests");
			sheet->writeStr(j, 4, "Failed requests");
			sheet->writeStr(j, 5, "Total transferred");
			sheet->writeStr(j, 6, "Requests per second");
			sheet->writeStr(j, 7, "Time per requests(user)");
			sheet->writeStr(j, 8, "Time per requests(server)");
			j++;
			while (getline(ifile, temp))
			{
				if (temp[0] == '/'){
					f << temp << " ";
					sheet->writeStr(j, i, temp.c_str());
				}
				else if (temp.find('[') != string::npos){
	
					f << temp.substr(0, temp.find('[') - 1) << " ";
					sheet->writeStr(j, i, temp.substr(0, temp.find('[') - 1).c_str());
				}
				else if (temp.find('b') != string::npos){
	
					f << temp.substr(0, temp.find('b') - 1) << " ";
					sheet->writeStr(j, i, temp.substr(0, temp.find('b') - 1).c_str());
				}
				else if (temp.find('s') != string::npos){
					sheet->writeStr(j, i, temp.substr(0, temp.find('s') - 1).c_str());
					f << temp.substr(0, temp.find('s') - 1) << " ";
				}
				else{
					sheet->writeStr(j, i, temp.c_str());
					f << temp << " ";
				}
				i++;
				if (i == 9){
					f << " " << endl;
					i = 0;
					j++;
				}
			}
			ifile.close();
		}
		if (book->save(argc[2])) //Save to example.xls
		{
			//.....
		}
		else
		{
			std::cout << book->errorMessage() << std::endl;
		}
		book->release();}
	return 0;
}

4 Some problems encountered during testing

When using a cloud host, you should pay attention to the bandwidth of the cloud host. Small pipes may become a bottleneck.

There is a discrepancy between Total transferred and port traffic in the ab software. The port traffic is greater than Total transferred, and it is speculated that there is a packet blocking factor. Therefore, Total transferred cannot be treated as traffic consumed by the server and used to calculate certain services on the cloud that consume traffic.

git: https://github.com/CollapsarLi/server_apachebench_shell.git

The above is the full content of this article. I hope it will be helpful for everyone’s study. I also hope that everyone will support 123WORDPRESS.COM.