Detailed explanation of how to build a CDN server with Nginx (picture and text)
|
Using Nginx's proxy_cache to build a cache server 1: Compile ngx_cache_purge 1. Nginx's Proxy_cache stores cache based on the Key value md5 hash and supports any Key. For example, you can combine "domain name, URI, parameters" into a key. It also supports non-200 status codes, such as 404/302, etc. 2. To use Nginx's Proxy_cache, you need to compile the ngx_cache_purge module into Nginx. Run: nginx -V to check if there is the word ngx_cache_purge. If not, you need to compile it manually.
3. Here, we use Oneinstack to compile the ngx_cache_purge module as a demonstration. If you are using other LNMP packages, you can refer to them. The basic process is similar. The command is as follows:
cd /root/oneinstack/src #Enter the installation package directory nginx -V
tar xzf nginx-1.10.3.tar.gz #Select the decompression package according to the nginx version checked above wget http://labs.frickle.com/files/ngx_cache_purge-2.3.tar.gz
tar zxvf ngx_cache_purge-2.3.tar.gz
cd /root/oneinstack/src/nginx-1.10.3
# For the parameters added after ./configure below, you can directly copy the parameters just obtained by nginx -V, and then add –add-module=../ngx_cache_purge-2.3 at the end, for reference:
./configure –prefix=/usr/local/nginx –user=www –group=www –with-http_stub_status_module –with-http_v2_module –with-http_ssl_module –with-http_gzip_static_module –with-http_realip_module –with-http_flv_module –with-http_mp4_module –with-openssl=../openssl-1.0.2k –with-pcre=../pcre-8.39 –with-pcre-jit –with-ld-opt=-ljemalloc –add-module=../ngx_cache_purge-2.3
make
mv /usr/local/nginx/sbin/nginx{,$(date +%m%d)}
cp objs/nginx /usr/local/nginx/sbin #oneinstack, other operations do not need this operation nginx -t
service nginx restart4. After the installation is complete, run nginx -V again and you will see that Nginx has been successfully compiled into ngx_cache_purge.
2. Use Nginx's proxy_cache to build a cache server 2: Modify the Nginx configuration file 1. First find your Nginx configuration file: nginx.conf (the path is usually /usr/local/nginx/conf/nginx.conf), and add the following code to the configuration file Http: (Note that the path should be changed to your own path) proxy_connect_timeout 5; proxy_read_timeout 60; proxy_send_timeout 5; proxy_buffer_size 16k; proxy_buffers 4 64k; proxy_busy_buffers_size 128k; proxy_temp_file_write_size 128k; proxy_cache_path /data/wwwroot/pic.freehao123.com levels=1:2 keys_zone=cache_one:200m inactive=30d max_size=5g; proxy_temp_path /data/wwwroot/pic.freehao123.com/temp; 2. The operation is as shown below: 3. Then in your virtual host's nginx.conf (the path is usually /usr/local/nginx/conf/vhost/pic.freehao123.com.conf), add the following commands to server listen 80 and listen 443 ssl http2:
location /{
proxy_pass https://www.freehao123.com;
proxy_redirect off;
proxy_set_header Host www.freehao123.com;
proxy_cache cache_one;
proxy_cache_valid 200 302 304 365d;
proxy_cache_valid 301 1d;
proxy_cache_valid any 1m;
add_header Images-Cache "$upstream_cache_status from $host";
add_header Pragma public;
add_header Cache-Control "public, must-revalidate, proxy-revalidate";
access_log off; log_not_found off; expires max;
}4. Save the configuration file and upload it again, then execute: nginx -t service nginx restart 5. First check whether the Nginx configuration is correct. If there is no problem, restart Nginx.
6. If you want to cache the gravatar avatar, the code is:
location /avatar{
proxy_pass http://cn.gravatar.com;
proxy_redirect off;
proxy_set_header Host cn.gravatar.com;
proxy_cache cache_one;
proxy_cache_valid 200 302 304 365d;
proxy_cache_valid 301 1d;
proxy_cache_valid any 1m;
add_header Images-Cache "$upstream_cache_status from $host";
add_header Pragma public;
add_header Cache-Control "public, must-revalidate, proxy-revalidate";
access_log off; log_not_found off; expires max;
}7. Now open your second-level domain name: pic.freehao123.com, and you can see that the image has been cached correctly.
8. Here is another Nginx cache code, which has the same effect as above.
#First write the following command in the Nginx configuration:
proxy_temp_file_write_size 128k;
proxy_temp_path /data/wwwroot/pic.ucblog.net/temp;
proxy_cache_path /data/wwwroot/pic.ucblog.net levels=1:2 keys_zone=cache_one:500m inactive=7d max_size=5g;
#Then write the following command in the Nginx configuration of the virtual host:
First add the following before the server listen 80 and listen 443 codes:
upstream gravatar {
server secure.gravatar.com:443;
}
#Add the following to server listen 80 and listen 443:
location / {
proxy_pass_header Server;
proxy_set_header Host cn.gravatar.com;
proxy_set_header Accept-Encoding ";
proxy_redirect off;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Scheme $scheme;
proxy_pass https://gravatar;
proxy_cache cache_one;
proxy_cache_valid 200 304 365d;
proxy_cache_key $host$uri$is_args$args;
expires max;
}9. On the VPS host, you can see the hash file generated by proxy_cache, which means that the caching has been successful.
3. Use Nginx's proxy_store to build a mirror server: modify Nginx configuration method 1. The function of Nginx's proxy_store is to directly create and read static files on the local hard disk, which is similar to the mirror CDN function of Qiniu or Youpai. The static images and other files of the source site will be automatically obtained during the first visit. Subsequent visits will be directly read from the CDN server, which speeds up the process. 2. Directly modify the Nginx virtual host configuration file (here img.freehao123.com.conf is used as a demonstration) and add the following code:
location / {
expires 3d;
proxy_set_header Accept-Encoding ";
root /data/wwwroot/img.freehao123.com;
proxy_store on;
proxy_store_access user:rw group:rw all:rw;
proxy_temp_path /data/wwwroot/img.freehao123.com/temp;
if ( !-e $request_filename ) {
proxy_pass https://www.freehao123.com;
}
}3. Save the configuration upload again, then restart Nginx. You can see that the static files such as images requested by img.freehao123.com have been successfully obtained from the source site.
4. In the storage directory on the VPS host, you can also see that proxy_store has completely saved the directory of static files such as pictures, which is equivalent to a mirror storage CDN of a website.
5. Here is another use, the effect is the same as above, remember to replace the path, the code is as follows:
upstream http_tornado {
server www.freehao123.com:443;
}
server {
#Omit other configuration location ~ .*\.(gif|jpg|jpeg|png|bmp|swf|js|html|htm|css)$ {
root /opt/data/product/blog/cache;
proxy_store on;
proxy_store_access user:rw group:rw all:rw;
proxy_temp_path /opt/data/product/blog/cache;
if ( !-e $request_filename ) {
proxy_pass http://http_tornado;
}
}
}4. What is the difference between Nginx’s proxy_store and proxy_cache? 1. The difference between mirroring and caching. From the above introduction, we can also see that proxy_store is equivalent to mirroring a website. The second access to static files such as pictures is directly read from the CDN server, which greatly reduces the burden on the source site. Proxy_cache is equivalent to caching, that is, generating a key for the request, so that the second access can be faster.
2. proxy_store is suitable for static, proxy_cache is suitable for dynamic. Proxy_store stores the entire image on the CDN server, so it is more suitable for image CDN acceleration, while proxy_cache generates the key for cache, which is more suitable for dynamic website acceleration and can be used for load balancing to reduce the burden on the server.
5. What to do after setting up a mirror CDN server? 1. First, because building a mirror CDN server completely copies the files and URLs of the source site, in order to avoid being mistaken by search engines as a plagiarized duplicate site, we can add Robots.txt to the CDN site to prevent search engines from including it. The command is as follows (pictures are allowed to be included, but others are not allowed to be crawled): User-agent: Baiduspider Allow: /wp-content/uploads/*.jpg$ Allow: /wp-content/uploads/*.png$ Allow: /wp-content/uploads/*.gif$ Disallow: / User-agent: 360Spider Allow: /wp-content/uploads/*.jpg$ Allow: /wp-content/uploads/*.png$ Allow: /wp-content/uploads/*.gif$ Disallow: / User-agent: Baiduspider-image Allow: /wp-content/uploads/*.jpg$ Allow: /wp-content/uploads/*.png$ Allow: /wp-content/uploads/*.gif$ Disallow: / User-agent: 360Spider-Image Allow: /wp-content/uploads/*.jpg$ Allow: /wp-content/uploads/*.png$ Allow: /wp-content/uploads/*.gif$ Disallow: / User-agent: Sosospider Allow: /wp-content/uploads/*.jpg$ Allow: /wp-content/uploads/*.png$ Allow: /wp-content/uploads/*.gif$ Disallow: / User-agent: sogou spider Allow: /wp-content/uploads/*.jpg$ Allow: /wp-content/uploads/*.png$ Allow: /wp-content/uploads/*.gif$ Disallow: / User-agent: YodaoBot Allow: /wp-content/uploads/*.jpg$ Allow: /wp-content/uploads/*.png$ Allow: /wp-content/uploads/*.gif$ Disallow: / User-agent: Googlebot Allow: /wp-content/uploads/*.jpg$ Allow: /wp-content/uploads/*.png$ Allow: /wp-content/uploads/*.gif$ Disallow: / User-agent: Bingbot Allow: /wp-content/uploads/*.jpg$ Allow: /wp-content/uploads/*.png$ Allow: /wp-content/uploads/*.gif$ Disallow: / User-agent: Slurp Allow: /wp-content/uploads/*.jpg$ Allow: /wp-content/uploads/*.png$ Allow: /wp-content/uploads/*.gif$ Disallow: / User-agent: MSNBot Allow: /wp-content/uploads/*.jpg$ Allow: /wp-content/uploads/*.png$ Allow: /wp-content/uploads/*.gif$ Disallow: / User-agent: googlebot-image Allow: /wp-content/uploads/*.jpg$ Allow: /wp-content/uploads/*.png$ Allow: /wp-content/uploads/*.gif$ Disallow: / User-agent: googlebot-mobile Allow: /wp-content/uploads/*.jpg$ Allow: /wp-content/uploads/*.png$ Allow: /wp-content/uploads/*.gif$ Disallow: / User-agent: yahoo-blogs/v3.9 Allow: /wp-content/uploads/*.jpg$ Allow: /wp-content/uploads/*.png$ Allow: /wp-content/uploads/*.gif$ Disallow: / User-agent: psbot Allow: /wp-content/uploads/*.jpg$ Allow: /wp-content/uploads/*.png$ Allow: /wp-content/uploads/*.gif$ Disallow: / User-agent: * Disallow: / 2. Second, do a good job of Nginx anti-hotlinking. If your CDN server traffic is not enough, it is recommended that you take anti-hotlink measures, which can also help you reduce the burden on the server. Add the following code to your virtual host configuration file:
location ~ .*\.(gif|jpg|jpeg|png|bmp|swf)$
{
valid_referers none blocked freehao123.com *.freehao123.com *.google.cn *.google.com *.google.com.hk image.baidu.com *.baidu.com;
if ($invalid_referer) {
rewrite ^/ https://www.freehao123.com;
#return 403;
}
}3. Third, set the Nginx default image. This is mainly for caching Gravatar avatars. When a certain image or file does not exist on the source server, we can set a default image or link for Nginx so that the cache looks perfect.
location /avatar {
try_files $uri /avatar/set-avatar.png;
}
#Or use:
location /{
try_files $uri /set-avatar.png;
}4. The effect is shown in the figure below:
This is the end of this article on how to use Nginx to build a CDN server (with pictures and text). For more information about using Nginx to build a CDN server, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope you will support 123WORDPRESS.COM in the future! You may also be interested in:
|
<<: How to use css variables in JS
>>: Examples of preview functions for various types of files in vue3
Recommend
Native js to realize a simple snake game
This article shares the specific code of js to im...
Installation of various versions of MySQL 8.0.18 and problems encountered during installation (essence summary)
Summary: Problem solving records of MYSQL: No mat...
Example method of viewing IP in Linux
Knowing the IP address of a device is important w...
The difference between mysql outer join and inner join query
The syntax for an outer join is as follows: SELEC...
Detailed explanation of Vue + Axios request interface method and parameter passing method
Table of contents 1. Get request: 2. Post request...
How to solve the problem that the software package does not exist when installing software in Linux
When the software package does not exist, it may ...
Introduction to the process of creating TCP connection in Linux system
Table of contents Steps to create TCP in Linux Se...
Detailed analysis of SQL execution steps
Detailed analysis of SQL execution steps Let'...
Detailed explanation of how to enter and exit the Docker container
1 Start the Docker service First you need to know...
Basic usage of wget command under Linux
Table of contents Preface 1. Download a single fi...
Several common ways to deploy Tomcat projects [tested]
1 / Copy the web project files directly to the we...
Detailed explanation of the use of MySQL sql_mode
Table of contents Preface sql_mode explained The ...
HTML Table Tag Tutorial (47): Nested Tables
<br />In the page, typesetting is achieved b...
Detailed usage of MYSQL row_number() and over() functions
Syntax format: row_number() over(partition by gro...
Example of customizing the style of the form file selection box
Copy code The code is as follows: <!DOCTYPE ht...