Implementation of nginx flow control and access control

nginx traffic control

Rate-limiting is a very useful, but often misunderstood and misconfigured feature in Nginx. We can use it to limit the number of HTTP requests a user can make in a given time. The request can be a GET request for a simple website homepage or a POST request for a login form. Rate limiting can be used for security purposes, such as to slow down the rate of brute force password cracking. It can also be used to protect against DDOS attacks by limiting the rate of incoming requests to a typical value for real users and identifying the target URL addresses (via logs). More commonly, this feature is used to protect upstream application servers from being overwhelmed by too many simultaneous user requests.

The following will introduce the basics and advanced configuration of Nginx traffic limiting. "Traffic limiting" also applies to Nginx Plus.

1. How to limit the flow of Nginx

Nginx's "flow limit" uses the leaky bucket algorithm, which is widely used in communications and packet switching computer networks to handle emergencies when bandwidth is limited. It's like a bucket with water pouring out of the mouth but leaking from the bottom. If the rate at which water is poured out of the bucket is greater than the rate at which water leaks out of the bottom of the bucket, the water in the bucket will overflow; similarly, in terms of request processing, water represents requests from the client, and the bucket represents a queue of requests waiting to be processed according to the "first-in-first-out scheduling algorithm" (FIFO). Water leaking from the bottom of the bucket represents requests that leave the buffer and are processed by the server, and water overflowing from the bucket represents requests that are discarded and not processed.

2. Configure basic current limiting

The "traffic limit" configuration has two main directives, limit_req_zone and limit_req , as shown below:

192.168.62.155 configuration:
limit_req_zone $binary_remote_addr zone=mylimit:10m rate=1r/s;
upstream myweb {
    server 192.168.62.157:80 weight=1 max_fails=1 fail_timeout=1;
    }
server {
    listen 80;
    server_name localhost;

    location /login {
        limit_req zone=mylimit;
        proxy_pass http://myweb;
        proxy_set_header Host $host:$server_port;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        }
}

192.168.62.157 configuration:
server {
    listen 80;
    server_name localhost;
    location /login {
        root /usr/share/nginx/html;
        index index.html index.html;
        }
}

After clicking twice

The limit_req_zone directive defines the parameters related to rate limiting, while limit_req directive enables rate limiting in the context in which it appears (in this case, for all requests to "/login/").

The limit_req_zone directive is usually defined in the HTTP block, making it usable in multiple contexts. It requires the following three parameters:

• Key - The request attribute that defines the applied restriction. The Nginx variable $binary_remote_addr in the example stores the binary form of the client IP address. This means that we can limit each different IP address to the request rate set by the third parameter. (This variable is used because it takes up less space than the string form of the client IP address $remote_addr )
• Zone - defines a shared memory zone used to store the status of each IP address and the frequency of access to the requested URLs that are restricted. The information stored in the shared memory area is meant to be shared between Nginx worker processes. The definition consists of two parts: the name of the zone identified by zone=keyword , and a colon followed by the zone size. The state information of 16,000 IP addresses requires about 1 MB, so the area in the example can store 160,000 IP addresses.
• Rate - Defines the maximum request rate. In the example, the rate cannot exceed 1 request per second. Nginx actually tracks requests at millisecond granularity, so the rate limit is equivalent to 1 request every 1000 milliseconds. Because "bursting" is not allowed (see the next section), this means that requests that arrive within 1000 milliseconds of a previous request will be rejected.

limit_req_zone directive sets the traffic limit and parameters of the shared memory zone, but does not actually limit the request rate. So you need to add

The limit_req directive applies traffic limits to a specific location or server block. In the above example, we limit the flow of /login/ requests.

Each IP address is now limited to one request to /login/ per second, or more precisely, it cannot request that URL within 1000 milliseconds of the previous request.

3. Handling emergencies

What if we receive 2 requests within 1000 milliseconds? For the second request, Nginx will return an error to the client. This may not be what we want, as applications tend to be bursty in nature. Instead, we want to buffer any excess requests and then process them in a timely manner. Let's update the configuration and use the burst parameter in limit_req :

limit_req_zone $binary_remote_addr zone=mylimit:10m rate=10r/s;
    upstream myweb {
        server 192.168.62.157:80 weight=1 max_fails=1 fail_timeout=1;
        }
       
    server {
        listen 80;
        server_name localhost;
        location /login {
            limit_req zone=mylimit burst=20;
            proxy_pass http://myweb;
            proxy_set_header Host $host:$server_port;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            }
    }

The burst parameter defines how many requests the client can make beyond the rate specified by the zone (in the example mylimit zone, the rate is limited to 10 requests per second, or one request every 100 milliseconds). Requests that arrive within 100 milliseconds of the previous request will be placed in the queue, and we set the queue size to 20.

This means that if 21 requests are sent from a given IP address, Nginx will immediately send the first request to the upstream server farm and then place the remaining 20 requests in a queue. It then forwards a queued request every 100 milliseconds, and only when an incoming request causes the number of requests queued in the queue to exceed 20 will Nginx return an error to the client.

4. Configure flow control related functions

1. Configure logging

By default, Nginx logs requests that are delayed or dropped due to rate limiting, as shown below:

2019/02/13 04:20:00 [error] 120315#0: *32086 limiting requests, excess: 1.000 by zone "mylimit", client: 192.168.1.2, server: nginx.com, request: "GET / HTTP/1.0", host: "nginx.com"

Fields included in the log entry:

• limiting requests - indicates that the log entry is for a request that is "limited"
• excess - the number of requests per millisecond that exceed the corresponding "rate limit" configuration
• zone - defines the zone where the "flow limit" is implemented
• client - the IP address of the client making the request
• server - server IP address or hostname
• request - the actual HTTP request initiated by the client
• host - the value of the host in the HTTP header
  

By default, Nginx logs rejected requests at the error level, as shown in [error] in the example above (Nginx logs delayed requests at a lower level, usually info level). To change Nginx's logging level, use the limit_req_log_level directive. Here, we set the logging level for rejected requests to warn :

Be sure to define the log location and level:

limit_req_zone $binary_remote_addr zone=mylimit:10m rate=1r/s;
    upstream myweb {
        server 192.168.62.157:80 weight=1 max_fails=1 fail_timeout=1;
        }
    server {
        listen 80;
        server_name localhost;

        location /login {
            limit_req zone=mylimit burst=20;
            limit_req_log_level warn;
            proxy_pass http://myweb;
            proxy_set_header Host $host:$server_port;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            }
    }

Continue to access the test and see the error.log log

2. Error code sent to the client

Normally, when the client exceeds the configured traffic limit, Nginx responds with a status code of 503 (Service Temporarily Unavailable). You can use the limit_req_status directive to set it to other status codes (such as the 404 status code below):

limit_req_zone $binary_remote_addr zone=mylimit:10m rate=10r/s;
	upstream myweb {
    	server 192.168.62.157:80 weight=1 max_fails=1 fail_timeout=1;
		}
	server {
    	listen 80;
    	server_name localhost;
		
    	location /login {
			limit_req zone=mylimit;
			limit_req_log_level warn;
			limit_req_status 404;
        	proxy_pass http://myweb;
          proxy_set_header Host $host:$server_port;
	      	proxy_set_header X-Real-IP $remote_addr;
      		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        	}
	}

5. Summary of nginx traffic control

This has covered many of the features of the "rate limiting" function provided by Nginx and Nginx Plus, including setting request rates for different locations of HTTP requests and configuring burst parameters for "rate limiting".

nginx access control

1. nginx access control module

(1) IP-based access control: http_access_module
(2) Login based on user trust: http_auth_basic_module

2. IP-based access control

1. Configuration syntax

Syntax: allow address | CIDR | unix: | all;
default: No Context by default: http, server, location

Syntax: deny address | CIDR | unix: | all;
default: No Context by default: http, server, location

2. Modify the content of /etc/nginx/conf.d/access_mod.conf as follows:

server {
    listen 80;
    server_name localhost;
    location ~ ^/admin {
        root /home/www/html;
        index index.html index.hml;
        deny 192.168.1.8;
        allow all;
        #deny 192.168.1.8;
        }
}
#Note:
If you allow access first, then define how to deny access. Then the access denial has no effect.

The virtual machine host IP is 192.168.1.8 , and the virtual machine IP is 192.168.1.11 , so the host machine is prohibited from accessing here, and all other IPs are allowed to access.
The host machine accesses http://192.168.1.11/admin and displays 403 Forbidden .
Of course, the reverse configuration is also possible, and the IP segment configuration method can also be used, such as allow 192.168.1.0/24; which means that all IPs that meet this segment can access it.

3. Specify location to reject all requests

If you want to deny all requests to a specific URL address instead of just limiting its rate, just configure deny all directive in location block:

server {
    listen 80;
    server_name localhost;
    location /foo.html {
        root /home/www/html;
        deny all;
        }
}

3. Login based on user trust

1. Configuration syntax

Syntax: auth_basic string | off;
default:auth_basic off;
Context: http, server, location, limit_except

Syntax: auth_basic_user_file file;
default: No context by default: http, server, location, limit_except
file: The file that stores username and password information.

2. Configuration Example

Rename access_mod.conf to auth_mod.conf and the content is as follows:

server {
	listen 80;
	server_name localhost;
	location ~ ^/admin {
		root /home/www/html;
		index index.html index.hml;
		auth_basic "Auth access test!";
		auth_basic_user_file /etc/nginx/auth_conf;
		}
}

auth_basic is not off , the login verification function is enabled, auth_basic_user_file loads the account and password file.

3. Create a password file

[root@192 ~]# mkdir /home/www/html/admin -p
[root@192 ~]# vim /home/www/html/admin
hello qf
[root@192 ~]# yum install -y httpd-tools #htpasswd is a command tool for the open source http server apache httpd, which is used to generate a password file for http basic authentication [root@192 ~]# htpasswd -cm /etc/nginx/auth_conf user10 //First time creating a new user [root@192 ~]# htpasswd -m /etc/nginx/auth_conf user20 //Second time adding a user [root@192 ~]# cat /etc/nginx/auth_conf
user10:$apr1$MOa9UVqF$RlYRMk7eprViEpNtDV0n40
user20:$apr1$biHJhW03$xboNUJgHME6yDd17gkQNb0

4. Access test

5. Limitations

(1) User information relies on files (2) Mechanical operation and management, low efficiency

This is the end of this article about the implementation of nginx flow control and access control. For more relevant nginx flow control and access control content, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope you will support 123WORDPRESS.COM in the future!