Detailed explanation of the implementation process of Nginx enabling Brotli compression algorithm

Preface

In web applications, in order to save traffic, reduce the size of transmitted data, and improve transmission efficiency, the commonly used compression method is generally gzip. Today we will introduce another more efficient compression method brotli.

Brotli is based on a modern variant of the LZ77 algorithm, Huffman coding, and second-order context modeling. Google software engineers released an enhanced version of Brotli in September 2015 that includes general lossless data compression, with a particular focus on HTTP compression.

Note: The premise of using the algorithm is to enable https, because the Accept-Encoding: gzip, deflate in the request header of the http request does not have br.
For more information about the Brotli algorithm, please see: https://link.zhihu.com/?target=https%3A//en.wikipedia.org/wiki/Brotli

Browser support for the brotli protocol

Comparison of various compression algorithms at different levels

From the figure, we can see that brotli vs gzip compression algorithms generally have better overall performance, especially the decompression speed. When we choose brotli algorithm or gzip, we need to tune it according to the actual scenario.

Download Brotli

google/ngx_brotli Since the December 2016 version, google/brotli has been built-in, so we don't need to compile the bagder/libbrotli library separately, making the installation simpler. We download and unzip google/ngx_brotli to the /usr/src/ngx_brotli directory

cd /usr/src

git clone https://github.com/google/ngx_brotli.git

Then download google/brotli and unzip it to /usr/src/ngx_brotli/deps/brotli

cd /usr/src/ngx_brotli/deps && rm -rf brotli
git clone [email protected]:google/brotli.git
cd /usr/src/ngx_brotli && git submodule update --init

Compiling Brotli

Nginx supports dynamic modules since version 1.9.11. Since then, you no longer need to recompile nginx to add modules to nginx. With dynamic modules, you can selectively load third-party or Nginx official modules at runtime. The new implementation remains as backwards compatible as possible via the API module.

Download and decompress the nginx installation package

Please download the nginx installation package that is the same as the current nginx version. The official download address of nginx is: http://nginx.org/en/download.html. Here it is assumed that the current server nginx is version 1.14.2.

You can get the current nginx version through the command

nginx -v

Output

nginx version: nginx/1.14.2

Download the nginx installation package

cd /usr/src
wget http://59.80.44.46/nginx.org/download/nginx-1.14.2.tar.gz

Unzip the installation package

tar -xvf nginx-1.14.2.tar.gz

Compiling dynamic modules

First enter the directory of the unzipped nginx installation package, configure, and then use make modules.

cd nginx-1.14.2
./configure --with-compat --add-dynamic-module=/usr/src/ngx_brotli
make modules

Parameter syntax: --add-dynamic-module=[absolute path to the directory where the module source code is located]

After the run is complete, check the compiled module

ls objs/*.so

Output:

objs/ngx_http_brotli_filter_module.so objs/ngx_http_brotli_static_module.so

Copy the compiled module file to the nginx dynamic module loading directory

cp objs/{ngx_http_brotli_filter_module.so,ngx_http_brotli_static_module.so} /etc/nginx/modules

Registering Brotli Modules

In order to facilitate the management of nginx dynamic modules, it is recommended to create a modules.conf file to manage dynamic modules separately.

touch /etc/nginx/modules.conf

Introduce the modules.conf file in the /etc/nginx/nginx.conf configuration file, find the following content and modify it:

pid /var/run/nginx.pid;

include /etc/nginx/modules.conf;

Open /etc/nginx/modules.conf and register the Brotli module just compiled.

Brotli Module
load_module modules/ngx_http_brotli_filter_module.so;
load_module modules/ngx_http_brotli_static_module.so;

Enable Brotli Compression

Brotli and gzip can coexist, there is no need to disable gzip.

Enable Brotli in /etc/nginx/nginx.conf:

http {
  ...
  # gzip
  gzip on;
  gzip_min_length 1k;
  gzip_buffers 4 32k;
  gzip_http_version 1.1;
  gzip_comp_level 5;
  gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/javascript;
  gzip_vary on;
  gzip_proxied any;
  gzip_disable "MSIE [1-6]\.";

  #brotli
  brotli on;
  brotli_comp_level 6;
  brotli_buffers 16 8k;
  brotli_min_length 20;
  brotli_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/javascript image/svg+xml;
  ...
}

Configuration details of gzip and brotli

Gzip tuning Using the gzip compression function may save us bandwidth, speed up transmission, provide a better experience, and save us costs, so this is a key point. For a detailed introduction to gzip, click here;

gzip on turns on gzip compression.

gzip_min_length 1k

Set the minimum number of bytes allowed for compressed pages. The number of bytes for a page is obtained from the Content-Length header. The default value is 0. No matter how large the page is, it will be compressed. It is recommended to set it to be greater than 1K. If it is less than 1K, the page size may increase as it is compressed.

gzip_buffers

The compression buffer size indicates applying for 4 units of 32KB of memory as the compression result stream cache. The default value is to apply for the same memory space as the original data size to store the gzip compression result.

gzip_http_version 1.1

The compressed version is used to set the HTTP protocol version. The default is 1.1. Currently, most browsers already support GZIP decompression, so you can use the default.

gzip_comp_level 5

Compression ratio is used to specify the gzip compression ratio. 1 means the smallest compression ratio and the fastest processing speed, while 9 means the largest compression ratio and fast transmission speed, but slow processing and more CPU consumption. The recommended setting is 5.

gzip_types

Used to specify the type of compression. The text/html type will always be compressed.

gzip_vary on

It is related to the http header. A vary header is added for the proxy server. Some browsers support compression, while others do not. In order to avoid wasting data, compression is performed on those that do not support it. Therefore, whether compression is needed is determined based on the client's HTTP header.

gzip_proxied any

Enabled when nginx is used as a reverse proxy, turn on or off the results returned by the backend server. The prerequisite for matching is that the backend server must return a header containing Via. The default is off. Optional parameter values:

• off Turn off compression of all proxy result data
• expired Enable compression if the header contains Expires header information
• no-cache enables compression if the header contains the Cache-Control: no-cache header information
• no-store enables compression if the header contains the Cache-Control: no-store header information
• private enables compression if the header contains the Cache-Control:private header information
• no_last_modified enables compression if the header does not contain the Last-Modified header information
• no_etag enables compression if the header does not contain ETag header information
• auth enables compression, if the header contains the Authorization header information
• any enables compression unconditionally

gzip_disable

Disable gzip compression for IE6. IE6's support for gzip compression is very poor, which may cause the page to freeze. To avoid problems with IE6, it is recommended to add this parameter.

brotli tuning

Google believes that Internet users' time is precious and should not be wasted on lengthy web page loading times, so in September 2015 Google launched the lossless compression algorithm Brotli. Brotli compresses data through a variant of the LZ77 algorithm, Huffman coding, and second-order text modeling. Compared with other compression algorithms, it has higher compression efficiency. For more information about brotli, please click here;

Note: If the brotli module is not installed, this part of the configuration items does not need to be configured and can be skipped.

Brotli on

Enable brotli compression.

brotli_comp_level 6

Compression ratio, used to specify brotli compression ratio, 1 is the smallest compression ratio and the fastest processing speed, 11 is the largest compression ratio, the transmission speed is fast, but the processing is slow and it consumes more CPU resources. The default value is 6, so you can use the default value.

brotli_buffers 16 8k

Sets the number and size of buffers used to compress responses. By default, the buffer size is equal to one memory page. Default value: 32 4k|16 8k.

brotli_min_length 20

Sets the minimum length of a response to be compressed. The length is determined solely by the Content-Length response header field. The default is 20.

brotli_types

Used to specify the type of compression. The text/html type will always be compressed.

The configuration details of gzip and brotli are added in the reverse proxy configuration file code:

proxy_set_header Accept-Encoding "";

example:

server {
  ...
  location / {
    ...
    proxy_set_header Accept-Encoding "";
    ...
  }
  ...
}

Restart nginx to make the configuration take effect

systemctl restart nginx

Brotli compression can only take effect in https because there is no br in the Accept-Encoding: gzip, deflate request header in the http request.
Cleaning up temporary files

You should develop a good habit of deleting the files or directories extracted from the application package after each compilation.

rm -rf /usr/src/{nginx-1.14.2/,ngx_brotli/}

Finally check if it works

Open the web page and debug it with Chrome developer tools. You will find content-encoding:br in the Network column, and the network time consumption will be significantly reduced.

The above is the full content of this article. I hope it will be helpful for everyone’s study. I also hope that everyone will support 123WORDPRESS.COM.