Some suggestions for improving Nginx performance

If your web application runs on only one machine, it is very simple to improve its performance: replace it with a faster one, add more processors, more memory, and a high-speed disk array. After the change, the WordPress server, Node.js or Java application running on this machine will be faster. (If the application also accesses another database server, that's also simple: find two faster machines and connect them with a faster network.)
The trouble is, machine speed isn't the issue. Many times, web applications are slow because they have to switch between various tasks, processing user requests on thousands of connections, reading and writing files to disk, running application code, and doing other things. The application server may therefore run into various situations, running out of memory, swapping files, or having many requests waiting for tasks such as a disk I/O. In addition to upgrading the hardware, you can actually choose a completely different approach: add a reverse proxy server to share some of the above tasks. The reverse proxy server is located in front of the machine running the application and is responsible for processing requests from the external network. The reverse proxy server is directly connected to the Internet, and it communicates with the application server using a fast internal network. The reverse proxy server allows the application server to focus on building pages, which are then sent to the reverse proxy to the external network without having to worry about the interaction between users and the application. Because the application server does not have to wait for a response from the client, it can run at near-optimal speeds.
Adding a reverse proxy server can also increase flexibility for your web server. For example, if a server performing a certain task is overloaded, you can always add another server of the same type; and if this server crashes, it is also easy to replace it. Given this flexibility, a reverse proxy server is often a prerequisite for other performance optimization techniques, such as:

• Load balancing (see "Suggestion 2"): Run a load balancing service on the reverse proxy server to evenly distribute traffic to several application servers. With load balancing, adding application servers does not require modifying the application at all.
• Cache static files (see "Recommendation 3"), files that can be directly requested, such as pictures or codes, can be stored in the reverse proxy server so that they can be sent directly to the client. This not only responds to requests faster, but also reduces the burden on the application server, making it run faster.
• To ensure site security, you can configure a reverse proxy server to improve its security level and use it to monitor and quickly identify and respond to attacks, thereby preserving the security of the application server.
  

NGINX is designed specifically for use as a reverse proxy server, making it naturally support the above optimizations. Because it uses event-driven processing, NGINX is more efficient than traditional servers. NGINX Plus adds more advanced reverse proxy features, such as application health checks, unique request routing, advanced caching, and after-sales support.

Suggestion 2: Add a load balancing server

Adding a load balancing server is relatively simple, but can significantly improve site performance and security. By distributing traffic to multiple servers, you don't have to upgrade your Web server. Even if the application itself is not well written or difficult to scale, load balancing can improve the user experience without making other changes.
The load balancing server is first a reverse proxy server (see "Suggestion 1"), which is responsible for forwarding requests from the Internet to other servers. The key here is that the load balancing server can support more than two application servers and use a selection algorithm to distribute requests among different servers. The simplest load balancing algorithm is round-robin scheduling, which forwards new requests to the next available server in turn. Other algorithms send requests to the server with the fewest active connections. NGINX Plus supports a feature called session persistence that keeps user sessions on the same server. Load balancing servers can prevent one server from being overloaded while others are too idle, thus greatly improving performance. At the same time, it can also make it easier to expand the capacity of the Web server, because you can choose cheaper servers while ensuring that they are fully utilized. The protocols that can be scheduled by load balancing include HTTP, HTTPS, SPDY, HTTP/2, WebSocket, FastCGI, SCGI, uwsgi, memcached, and some other application forms, including TCP-based applications and other fourth-layer protocols. To do this, you first need to analyze the web application to see where the performance shortcomings are, and then decide which one to use. The same server or servers used for load balancing can also take on other tasks, such as SSL termination, supporting HTTP/1/x or HTTP/2 depending on the client, and caching static files. NGINX is often used for load balancing. For more information, please refer to our previous introductory articles, configuration articles, e-books, related online videos, and of course the documentation. Our commercial version, NGINX Plus, supports more load balancing features, such as routing loads based on server response time and load balancing that supports the Microsoft NTLM protocol.

Tip 3: Cache static and dynamic content

Caching can improve web application performance by delivering content to clients faster. Caching strategies include pre-processing content, storing content on faster devices, keeping content close to the client, and a combination of these strategies. There are two types of cache.

• Static content caching: Infrequently changing files, such as images (JPEG, PNG) and code (CSS, JavaScript), can be stored in edge servers for quick retrieval from content or disk.
• Dynamic content caching. Many web applications generate new HTML for each page request. Caching each generated HTML for a short period of time may significantly reduce the total number of pages that need to be generated while ensuring that the delivered content is fresh enough.
  

Assume a page is viewed 10 times per second, and you cache it for 1 second, then 90% of the requests for this page will come from the cache. If you cache static content separately, then even a freshly generated page will likely be mostly served from cached content. There are three main techniques for caching content generated by web applications.

• Put the content close to the users. Close to the user, less transmission time.
• Put the content on a faster machine. The machine is fast and the retrieval speed is fast.
• Take content away from overused machines. Sometimes machines are much slower than when they are focused on a specific task because they are distracted by too many tasks. At this time, moving the content to other machines is not only beneficial for cached content, but also for non-cached content because the burden on the host hosting them is reduced.
  

Web application caching can be implemented inside or outside the Web application server. First, consider caching dynamic content to reduce the load on your application servers. Second, caching is used for static content (including temporary copies of dynamically generated content), further reducing the burden on application servers. Then, consider moving the cache to other machines that are faster or closer to the user to reduce the load on the application server and shorten the transmission time. Proper use of cache can significantly speed up the response of your application. For many web pages, static data such as large images often occupy more than half of the content. Without caching, querying and transferring this kind of data might take several seconds, but with caching, it might take just a fraction of a second. To illustrate how to use caching, NGINX and NGINX Plus set up caching with two directives: proxy_cache_path and proxy_cache, which specify the location and size of the cache, the maximum cache time, and other parameters. Using the third (and popular) directive proxy_cache_use_stale, you can even tell the cache to serve old files when the server that should have provided fresh content is too busy or down. For the client, getting the content is always better than not getting it at all. From the user's perspective, this can also establish the image that your site or application is very stable. NGINX Plus supports advanced caching features, including cache purging and a dashboard that visualizes cache status for real-time monitoring. To learn more about caching in NGINX, see NGINX Content Caching in the reference documentation and the NGINX Plus Admin Guide. Note: Caching involves development, decision-making, and operations. A complete caching strategy, such as the one mentioned in this article, can reflect the value of considering it from a DevOps perspective. In other words, developers, architects, and operations and maintenance personnel work together to ensure the functionality, response time, security, and business goals of a website.

Tip 4: Compress data

Compression can also greatly improve performance. There are very mature and efficient compression standards for pictures, videos, music and other files (JPEG and PNG, MPEG-4, MP3), any one of which can reduce the file size by an order of magnitude or more.
Text files, including HTML (plain text and HTML tags), CSS, and JavaScript code, are often transferred without compression. Compressing this data can sometimes significantly improve the perceived performance of a web application, especially for mobile users with slow and unreliable connections. Because text data can play a necessary supporting role in page interaction, while multimedia data is more of a icing on the cake. Smart content compression can reduce the size of text content such as HTML, JavaScript, CSS, etc. by more than 30%, thereby reducing loading time accordingly.
If you use SSL, compression can reduce the amount of data that must be SSL-encoded, thus making up for the CPU time spent compressing that data.
There are many ways to compress data. For example, the part about HTTP/2 in Recommendation 6 describes a novel compression idea that is particularly suitable for header data compression. As another example of text compression, you can enable GZIP compression in NGINX. After pre-compressing the text data, you can use the gzip_static directive to send the .gz file directly.

Tip 5: Optimize SSL/TLS

More and more websites are using Secure Sockets Layer (SSL) and later Transport Layer Security (TLS) protocols. SSL/TLS improves website security by encrypting data sent from the origin server to the user. Google will give this process a big boost by boosting the search engine rankings of sites that use SSL/TLS.
Despite growing adoption, the performance penalties caused by SSL/TLS also plague many websites. There are two reasons why SSL/TLS slows down your website. 1. The initial handshake that opens each new connection must create encryption keys, and the way browsers use HTTP/1.x to establish multiple connections to each 2. server further exacerbates this problem. The operations of encrypting data on the server side and decrypting data on the client side are also overhead. To encourage people to use SSL/TLS, the authors of HTTP/2 and SPDY (see Recommendation 6) designed these two protocols to allow browsers to only establish one connection per session. This eliminates one of the two main reasons why SSL can degrade performance. However, there is still a lot that can be done when it comes to optimizing SSL/TLS performance. Methods for optimizing SSL/TLS vary from web server to web server. Take NGINX as an example. NGINX uses OpenSSL and runs on ordinary machines, which can provide performance close to that of customized machines. NGINX SSL performance details how to minimize the overhead of SSL/TLS encryption and decryption. In addition, here is an article that introduces many ways to improve SSL/TLS performance. To summarize briefly, the main technologies involved are as follows.

• Session cache. Use the ssl_session_cache directive to enable caching and cache the parameters used for each SSL/STL connection.
• Session ticket or ID. Saves information about a specific SSL/TLS session as a session ticket or ID so that the connection can be reused without having to re-handshake.
• OCSP Envelope. Reduce handshake time by caching SSL/TLS certificate information.

Both NGINX and NGINX Plus can terminate SSL/TLS, which means handling the encryption and decryption of client-side information while maintaining clear text communications with other servers. Setting up NGINX or NGINX Plus to handle SSL/TLS termination can take several steps. For using NGINX Plus on a server that accepts TCP connections, follow the setup instructions here.

Recommendation 6: Implement HTTP/2 or SPDY

Sites that already use SSL/TLS are likely to see performance improvements if they use HTTP/2 or SPDY, since only one handshake is required for a connection. Sites that are not already using SSL/TLS, HTTP/2, and SPDY may see a regression in terms of responsiveness if they switch to SSL/TLS (which typically reduces performance).
Google started the SPDY project in 2012, dedicated to achieving faster speeds on top of HTTP/1.x. HTTP/2 is a standard recently approved by the IETF that is based on SPDY. SPDY is widely supported but will soon be replaced by HTTP/2.
The key to SPDY and HTTP/2 is to use only one connection instead of multiple connections. This single connection is multiplexed, so it can carry multiple requests and responses simultaneously.
Maintaining only one connection saves the setup and management overhead of multiple connections. And a connection is particularly important for SSL because it can minimize the handshake time required for SSL/TLS to establish a secure connection.
The SPDY protocol requires the use of SSL/TLS, which is not officially required for HTTP/2, but all browsers that currently support HTTP/2 will only use it when SSL/TLS is enabled. In other words, a browser that supports HTTP/2 will only use HTTP/2 if the website uses SSL and the server accepts HTTP/2 traffic. Otherwise, the browser will communicate based on HTTP/1.x.
After implementing SPDY or HTTP/2, previous performance optimization measures for HTTP, such as domain name sharding, resource merging, and image sprites, are no longer necessary. Therefore, the code and deployment can also be simplified. For more information about the changes HTTP/2 will bring, please refer to our white paper.

NGINX has supported SPDY for a long time, and most sites using SPDY today are running NGIN
X. NGINX was also the first to support HTTP/2. In September 2015, NGINX Open Source and NGINX Plus began to support HTTP/2.
Over time, NGINX expects most sites to enable SSL and migrate to HTTP/2. This not only makes the website more secure, but also allows for higher performance with simpler code as new optimization techniques emerge.

Recommendation 7: Upgrade software

A simple way to improve application performance is to select software based on reliability and performance. In addition, developers of high-quality components are more likely to continuously improve performance and fix issues, so it is cost-effective to use the latest stable version. New releases receive more attention from developers and users, and also take advantage of new compiler optimization techniques, including tuning for new hardware. Compared with the old version, the newly released stable version has obviously higher performance. By keeping up with upgrades, you can also stay up to date with tuning, problem fixes, and security alerts. Failure to upgrade software can also hinder taking advantage of new capabilities. For example, HTTP/2 currently requires OpenSSL 1.0.1. Starting in the second half of 2016, HTTP/2 will require OpenSSL 1.0.2, which was released in January 2015. NGINX users can start with the latest version of NGINX Open Source software or NGINX Plus, which supports socket sharing, thread pools (see below), and ongoing performance optimizations. So check your software and try to update them to the latest versions.

Tip 8: Tune Linux

Linux is the underlying operating system for most web servers today. As the foundation of all infrastructure, Linux is critical to improving performance. By default, many Linux systems are relatively conservative, with desktop office requirements as the only requirement and a small amount of resources as the tuning goal. For web applications, re-tuning is definitely necessary to achieve optimal performance. Linux optimization varies from web server to web server. Taking NGINX as an example, we can consider the following aspects.
Inventory queue. If you find that some connections are not being processed, you can increase net.core.somaxconn, which is the maximum number of connections waiting to be processed by NGINX. If the connection limit is too low, you should see error messages and can gradually increase this value until the error messages no longer appear.

• File descriptor. NGINX uses at most two file descriptors per connection. If the system serves many connections, you may need to increase the system-wide limit on descriptors, sys.fs.file_max, and the user file descriptor limit, nofile, to support the increased load.
• Ephemeral port. When used as a proxy, NGINX creates ephemeral ports for each upstream server. You can set net.ipv4.ip_local_port_range to increase the range of port values ​​to increase the amount of available ports. In addition, you can also reduce the value of net.ipv4.tcp_fin_timeout, which controls the wait time before inactive ports are released for reuse, to speed up turnover.
• For NGINX, refer to the NGINX Performance Tuning Guide to learn how to effortlessly optimize your Linux system to support greater throughput.
  

Tip 9: Tune your web server

No matter what web server you use, you need to tune it for your application. The following suggestions apply to any web server, but setup instructions are given for NGINX only.

• Access logs. Don't write every request log to disk immediately. You can cache them in memory and then batch them in. For NGINX, add the buffer=_size_ parameter to the access_log directive to wait until the memory buffer is full before writing the log to disk. If you add the **flush=_time_** parameter, the buffer contents will also be written to disk at the specified time.
• buffer. Buffering is used to store partial responses in memory until the buffer is filled, allowing for more efficient responses to clients. Responses that cannot fit into memory are written to disk, degrading performance. When NGINX buffering is enabled, it can be managed using the proxy_buffer_size and proxy_buffers directives.
• Client active connections. Active connections can reduce latency, especially when using SSL/TLS. For NGINX, you can increase the value of keepalive_requests for the client, which defaults to 100; you can also increase the value of keepalive_timeout to make active connections last longer, so that subsequent requests can be responded to more quickly.
• Upstream active connection. Upstream connections, that is, connections to application servers and database servers, can also benefit from the setting of active connections. For upstream connections, you can increase active connections, which is the number of idle active connections available to each worker process. This can improve connection reuse and reduce reopening connections. For more information about active connections, please refer to this blog post.
• limit. Limiting the resources used by clients can improve performance and security. For NGINX, the limit_conn and limit_conn_zone directives limit the number of connections from a specified source, while limit_rate limits bandwidth. These settings prevent legitimate users from "stealing" resources and also help prevent attacks. The limit_req and limit_req_zone directives limit client requests. For connections to upstream servers, you can use the max_conns parameter in the server directive in the upstream configuration area, which limits the number of connections to the upstream server to prevent overload. The related queue directive creates a queue that holds the specified number of requests for the specified time after the max_conns limit is reached.
• Work process. Worker processes are responsible for handling requests. NGINX uses an event-based model and OS-dependent mechanisms to efficiently distribute requests among worker processes. It is recommended to set the value of worker_processes to one worker process per CPU. Most systems will support raising the value of worker_connections (the default is 512) if necessary. Experiment to find the value that works best for your system.
• Socket sharding. Typically, a socket listener distributes new connections to all worker processes. Socket sharding creates a socket listener for each worker process, and the kernel assigns a connection to the socket listener when it is available. This can reduce lock contention and improve performance on multi-core systems. To enable socket sharding, include the reuseport parameter in the listen directive.
• Thread pool. A time-consuming operation can block any computer process. For Web server software, disk access can hinder many faster operations, such as in-memory calculations and replication. When using a thread pool, slow operations are assigned to a separate set of tasks while the main processing loop continues to run faster operations. Once the disk operation is complete, the results are returned to the main processing loop. In NGINX, the read() system call and sendfile() are offloaded to the thread pool.

Tip: When modifying any operating system or peripheral device settings, modify only one item at a time and then test the performance. If the change causes problems or does not improve performance, change it back.

Tip 10: Monitor real-time dynamics to identify problems and bottlenecks

The key to maintaining high application performance is to monitor application performance in real time. The dynamics of applications in specific devices and corresponding web infrastructure must be monitored in real time.
Monitoring site activity is mostly passive; it tells you what's happening, but it's up to you to find and fix problems.
Monitoring can capture the following problems: 1. Server downtime 2. Server instability, missed connection 3. Large-scale cache failure on the server 4. Incorrect content sent by the server
Global performance monitoring tools such as New Relic or Dynatrace can help us monitor the time it takes to load remote pages, while NGINX can help you monitor the application delivery end. Your app’s performance data can tell you when your optimizations are truly making a difference to your users, and when you need to scale up to accommodate increasing traffic.
To help users detect problems as quickly as possible, NGINX Plus has added an application health check feature that reports frequently occurring problems. NGINX Plus also has a session draining feature that blocks new connections until existing tasks are completed, as well as slow start capacity to allow recovered servers to get up to speed in a load-balanced cluster. When used properly, health checks can help you identify problems before they significantly impact the user experience, while session draining and slow start allow you to replace servers without affecting perceived performance and uptime. This image shows the dashboard for NGINX Plus’ built-in real-time activity monitoring, covering servers, TCP connections, and cache.

Conclusion: 10x performance improvement

The performance improvement will vary greatly from one web application to another. The actual improvement depends on budget, time, and the gap between the existing implementation and the ideal performance. So how can you get a 10-fold performance improvement for your application? To help you understand the potential of each optimization suggestion, we will provide some implementation guidelines for the previous suggestions. I hope everyone can take what they need.

• Reverse proxy server and load balancing. No load balancing or pooled load balancing may result in very poor performance. Adding a reverse proxy server, such as NGINX, can reduce the number of round trips your web application makes between memory and disk. Load balancing can transfer tasks from overloaded servers to idle servers, and also facilitates expansion. These changes can greatly improve performance. Compared with the worst case of the original deployment method, a 10-fold performance improvement is a very easy thing. Even if it is less than 10 times, it is a qualitative leap overall.
• Cache dynamic and static content. If your web server also acts as an application server, you can achieve a 10-fold performance improvement at peak times by caching dynamic content. Caching static content can also improve performance several times.
• Compress data. Using compression formats such as JPEG, PNG, MPEG-4, and MP3 can significantly improve performance. If all these measures are used, then compressed text data (code and HTML) can improve the initial page load time by two times.
• Optimize SSL/TLS. The security handshake has a huge impact on performance, so optimizing it can make initial responses up to twice as fast, especially for text-heavy sites. Optimizing media files for SSL/TLS provides only a small performance improvement.
• Implement HTTP/2 and SPDY. When using SSL/TLS, these two protocols have the potential to improve the overall performance of your website.
• Tuning Linux and web servers. Using optimized buffering strategies, using active connections, and offloading time-consuming tasks to independent thread pools can significantly improve performance. For example, thread pools can improve the performance of disk-intensive tasks by at least an order of magnitude.

The above are the details of some suggestions for improving Nginx performance. For more information on improving Nginx performance, please pay attention to other related articles on 123WORDPRESS.COM!