Solve the problem of Syn Flooding in MySQL database

Syn attack is the most common and most easily exploited attack method. It takes advantage of the defects of the TCP protocol to send a large number of forged TCP connection requests. A large number of SYN packets are often sent using fake IPs. The attacked server responds with SYN+ACK. Because the other party is a fake IP, it will never receive the packet and will not respond. As a result, the attacked server maintains a large number of semi-connections in the SYN_RECV state and will retry the default 5 response handshake packets, filling up the TCP waiting connection queue, exhausting resources, and preventing normal business requests from connecting.

Syn attacks are common on application servers, and database servers are in the intranet, so it is unlikely to encounter similar attacks. However, sometimes if the application is not connected to the database correctly, it will be considered a Syn attack on the database side and the connection will be rejected.

[Problem description]

The database suddenly refuses to connect, and the application reports an error. At the time of the problem, the following error message can be seen in the operating system log of the database server, that is, /var/log/messages:

kernel: possible SYN flooding on port 3306. Sending cookies.

【Problem Analysis】

At the point where the problem occurred, judging from the database monitoring indicators, the Threads Connected indicator increased. This is also very obvious, because for the database, Syn Flooding means that the application suddenly initiates a connection to the database, and the operating system cannot handle it, so it reports Syn Flooding. From the perspective of database performance indicators, the number of connections will definitely have a sudden increase. The solution is to analyze where these sudden increases come from, smooth out the peaks and fill the valleys, and make the connection more stable.

【Solution】

On the database server side, make the following adjustments: This adjustment means: increase the TCP half-connection buffer. The default value is 2048, and we adjust it to 8192 to increase the system's ability to withstand sudden pressure. The default value of Tcp_syn_retires and Tcp_synack_retires is 5, which means that the server needs to send five packets before terminating the retry. We adjust this parameter to 2. We only retry once, so that the error packet can be resolved as early as possible to reduce the number of cached connections.

echo 8192 > /proc/sys/net/ipv4/tcp_max_syn_backlog
echo 2 > /proc/sys/net/ipv4/tcp_syn_retries
echo 2 > /proc/sys/net/ipv4/tcp_synack_retries

This parameter adjustment takes effect immediately without restarting. Of course, after the server is restarted, these parameters will return to the default values. After this adjustment, the database's stress resistance was enhanced, but the problem was not completely solved.

We also make corresponding adjustments on the client side:

To reduce the pressure on the number of database connections, we usually recommend that the connection pool be configured as follows:

testWhileIdle="false". Do not check connection string health when idle
minIdle="0". The minimum number of idle connections in the connection pool
maxAge="30000". A link can be recycled after a certain number of milliseconds.
initialSize="1". The minimum number of initial connections in the connection pool
timeBetweenEvictionRunsMillis="5000". The running interval of the recycling thread (milliseconds)

For the current scenario, we recommend increasing the minIdle parameter from 0 to 5. Let the connection pool usually have 5 idle connections. In this way, when a request to the database is initiated, these 5 idle connections will be used first. To achieve the effect of reducing peaks and filling valleys. Of course, the side effect is that the number of database connections will increase. The appropriate adjustment amount needs to be based on the actual database connection load. For .NET programs, there are also corresponding connection pool parameters that can be adjusted: the minPoolSize parameter can be appropriately modified and also adjusted to 5.

After this adjustment, basically most of the database Syn Flooding problems can be solved.

Of course, these are just tuning methods that can only slightly improve the system. Improve stress resistance. The final analysis still depends on where the connection pressure comes from. And why a large number of connections to the database need to be established in bursts. Is it appropriate to use a database for this kind of emergency scenario? An alternative is to use Redis as a buffer in front. Avoid sudden connection requests to the database. This involves the transformation of the application.

Summarize

The above is the editor's introduction to solving the problem of Syn Flooding in MySQL database. I hope it will be helpful to everyone. If you have any questions, please leave me a message and I will reply to you in time. I would also like to thank everyone for their support of the 123WORDPRESS.COM website!
If you find this article helpful, please feel free to reprint it and please indicate the source. Thank you!