Detailed explanation of Nginx log customization and enabling log buffer

Preface

If you want to count the source of website visits, you can use PHP to obtain the information and record it in the database, or you can directly use the access log provided by nginx to record the website's visit details. Administrators can analyze the user's access source, access behavior details, website page access popularity, etc. by analyzing nginx's access log. In addition, nginx itself also has error logs, which makes it easier for operation and maintenance personnel to debug nginx. For the behavior of recording logs, if disk operations are performed every time, more resources will be consumed. Based on this situation, the nginx log buffer can be enabled. When the buffer is full or the scheduled writing time is reached, the log is written again.

Access log

nginx writes information about client requests in the access log immediately after processing the request. By default, the access log is located in logs/access.log and the information is written to the log in a predefined combined format.

To accurately record access information, you need to customize a more complete access log format, as shown below:

http {
  log_format geoproxy
  '[$time_local] $remote_addr '
  '$realip_remote_addr $remote_user '
  '$request_method $server_protocol '
  '$scheme $server_name $uri $status '
  '$request_time $body_bytes_sent '
  '$geoip_city_country_code3 $geoip_region '
  '"$geoip_city" $http_x_forwarded_for '
  '$upstream_status $upstream_response_time '
  '"$http_referer" "$http_user_agent"';
  ...
}

This logging configuration is named geoproxy and it uses a number of nginx variables to demonstrate nginx logging capabilities. Explain in detail the specific meaning of each variable in the configuration options:

When a user initiates a request, the server time $time_local is recorded, and the value of $remote_user is the user name that has passed basic authorization;

The IP address of the open connection and the client IP address used by nginx to process the geoip_proxy and realip_header directives;

Then record the HTTP request method $request_method, protocol $server_protocol and HTTP method $scheme: http or https;

Of course, there is also the server name $server_name , the requested URI, and the response status code;

In addition to basic information, there are also some statistical result data: including the millisecond time of request processing $request_time and the data block size of the server response $body_bytes_sent;

In addition, the client's country $geoip_city_country_code3 , region $geoip_region , and city information $geoip_city are also recorded;

The variable $http_x_forwarded_for is used to record the X-Forwarded-For header message of requests initiated by other proxy servers;

Some data from the upstream module is also recorded in the log: the response status code of the proxied server $upstream_status , the time between establishing the connection and receiving the last byte of the response body from the upstream server $upstream_response_time , the time between establishing the connection with the upstream server $upstream_connect_time , and the time between establishing the connection and receiving the first byte of the upstream response header $upstream_header_time .

The request source $http_referer and user agent $http_user_agent can also be recorded in the log;

nginx's logging function is very powerful and flexible. It should be noted that: the log_format directive used to define the log format is only applicable to http block-level directives, and all time values ​​are measured in milliseconds with millisecond resolution. .

This format of log configuration will generate the following types of logs:

[25/Feb/2019:16:20:42 +0000] 10.0.1.16 192.168.0.122 Derek
GET HTTP/1.1 http www.example.com / 200 0.001 370 USA MI
"Ann Arbor" - 200 0.001 "-" "curl/7.47.0"

If you need to use this log configuration, you need to use it in conjunction with the access_log directive, which receives a log directory and the configuration name to use as parameters:

server {
  access_log /var/log/nginx/access.log geoproxy;
  ...
}

access_log can be used in multiple contexts, each of which can define its own log directory and log record format.

Conclusion: The log module in nginx allows you to configure the log format for different scenarios so that you can view different log files.

In practice, it is very useful to configure different logs for different contexts. The log content can be simple information or detailed record of all necessary information. In addition to supporting text

It can also record data in json and xml formats. In fact, nginx logs can help you understand information such as server traffic, client usage, and client sources. In addition, access logs can also help you locate responses and problems related to upstream servers or specific URIs; access logs are also useful for testing, as they can be used to analyze traffic conditions and simulate real user interaction scenarios. Logs are indispensable in troubleshooting, debugging, application analysis, and business adjustments.

Error Log

In order to accurately locate the error log of nginx, use the built-in error_log directive to define the error log directory and the level of error log recording. The configuration is as follows:

error_log /var/log/nginx/error.log warn;

The error_log directive takes a required log directory and an optional error level option.

The error_log directive can be used in all contexts except the if directive. Error log levels include:

debug, info, notice, warn, error, crit, alert, and emerge. The log given

The level order is the order of log levels from the smallest to the most rigorous. It is important to note that the debug log

You need to compile the nginx server with the --with-debug flag to use it.

When a server configuration error occurs, you first need to check the error log to locate the problem. Error Log

It is also a powerful tool for locating application servers (such as FastCGI servers). Through the error log, we can debug problems such as worker process connection errors, memory allocation, client IP and application server. The error log format does not support custom log formats; but it also records data such as the current time, log level, and specific information.

Note: The default settings for error logging apply globally. To override this, place the error_log directive in the main (top-level) configuration context. The ability to specify multiple error_log directives at the same configuration level was added in NGINX Open Source version 1.5.2.

Send logs to the unified server via syslog

Since the logs no longer need to be written to a directory on the disk, but sent to a unified log server, just replace the original directory part with the server IP address. The configuration is as follows:

error_log syslog:server=10.0.1.42 debug;
access_log syslog:server=10.0.1.42,tag=nginx,severity=info geoproxy;

#error_log server=unix:/var/log/nginx.sock debug;
#access_log syslog:server=[2001:db8::1]:1234,facility=local7,tag=nginx,severity=info;

The syslog parameter to the error_log and access_log directives is followed by a colon : and some parameter options. Includes: The required server tag indicates the IP, DNS name or UNIX socket to connect to;

Can be played with high end as noted above.

Optional parameters are facility , severity , tag :

The server parameter accepts an IP address or DNS name with a port; the default is UDP port 514.

The facility parameter sets the syslog type facility. The value is one of the 23 values ​​defined by the syslog RFC standard. The default value is local7. Other possible values ​​are: auth , authpriv , daemon , cron , ftp , lpr , kern , mail , news , syslog , user , uucp , local0 ... local7

The tag parameter indicates the title displayed in the log file. The default value is nginx.

severity sets the message severity, the default is info level log.

Log Buffer

When the system is under load, enable log buffer to reduce nginx worker process blocking. Large amounts of disk reading and writing and CPU resource usage are also a huge consumption of server resources. Buffering log data into memory may be a small optimization method. The buffer parameter means the size of the buffer. When the buffer is full, the log will be written to the file. The flush parameter means the maximum time that the log in the buffer is kept in the buffer memory. When the log in the cache exceeds the maximum cache time, it will also be written to the file. The shortcoming is that there is a slight delay in writing the log to the log file. Log buffering should be turned off during real-time debugging. . The configuration is as follows:

http {
  access_log /var/log/nginx/access.log main buffer=32k flush=1m;
}

Reference Links:

1. nginx cookbook
2. nginx configuration log

Summarize

The above is the full content of this article. I hope that the content of this article will have certain reference learning value for your study or work. Thank you for your support of 123WORDPRESS.COM.