Deeply understand how nginx achieves high performance and scalability

The overall architecture of NGINX is characterized by a set of processes working together:

Main process: responsible for performing privileged operations, such as reading configuration files, binding sockets, and creating/signaling child processes.

Worker process: Responsible for receiving and processing connection requests, reading and writing to disk, and communicating with upstream servers. When NGINX is active, only the worker processes are busy.

Cache loader process: responsible for loading the disk cache into memory. This process runs at startup and then exits.

Cache manager process: responsible for organizing the disk cache data to ensure that it does not cross the boundary. This process will run intermittently.

The key to NGINX's high performance and scalability lies in two fundamental design choices:

Limit the number of worker processes as much as possible to reduce the overhead caused by context switching. The default and recommended configuration is to have one worker process per CPU core to efficiently utilize hardware resources.

The worker process is single-threaded and handles multiple concurrent connections in a non-blocking manner.

Each NGINX worker process handles multiple connection requests through a state machine that is implemented in a non-blocking manner:

Each worker process needs to handle several sockets, including listening sockets or connecting sockets.

When the listening socket receives a new request, a new connection socket is opened to handle the communication with the client.

When an event arrives for a connected socket, the worker process quickly completes the response and moves on to processing any newly received events for any other sockets.

Garrett said NGINX's design choices make it fundamentally different from other Web servers. A typical web server will choose to assign each connection to a separate thread, which makes it very easy to handle multiple connections because each connection can be considered as a linear sequence of multiple steps, but this will incur the overhead of context switching. In fact, worker threads spend most of their time blocked, waiting for clients or other upstream servers. When the number of concurrent connections/threads attempting to perform I/O operations exceeds a certain threshold, or when memory is exhausted, the cost of context switching becomes apparent.

NGINX, on the other hand, is designed not to let worker processes block network traffic unless there is no work to do. Furthermore, each new connection consumes very few resources, including only a file descriptor and a small amount of worker process memory.

In general, after the system is tuned, each of NGINX's working processes can handle hundreds or thousands of concurrent HTTP connections.

Inside NGINX: How We Designed It for Performance and Scalability

The reason why NGINX is so superior in performance is due to its design. While many web servers and application servers use simple threaded or process-based architectures, NGINX stands out with a sophisticated event-driven architecture that can support tens of thousands of concurrent connections on modern hardware.

The Inside NGINX infographic covers everything from a high-level exploration of the process architecture to a diagram of how a single NGINX process handles multiple connections. This article explains the details of these operations.

Setting the scene - the NGINX process model

Setting the Scene? the NGINX Process Model

To better understand the design, you need to understand how NGINX works. NGINX has a master process (which performs privileged operations such as reading configuration and binding to ports) and a series of worker processes and helper processes.

On this quad-core server, the NGINX master process creates four worker processes and two cache helper processes to manage the on-disk content cache.

Why is architecture important?

Why Is Architecture Important?

The fundamental basis of any Unix application is threads or processes. (From the Linux operating system's perspective, threads and processes are essentially the same; the main difference is the degree to which they share memory.) A process, or thread, is a set of independent instructions that the operating system can schedule to run on a CPU core. Most complex applications run multiple threads or processes in parallel for two reasons:

● More computer cores can be used simultaneously.

●Threads and processes make parallel operations easy to implement (for example, handling multiple connections simultaneously).

Both processes and threads consume resources. They all use memory and other OS resources, causing frequent core switching (an operation called a context switch). Most modern servers can handle hundreds of small, active threads or processes simultaneously, but once memory is exhausted or high I/O loads result in a large number of context switches, server performance can degrade severely.

For network applications, a thread or process is usually assigned to each connection. This architecture is easy to implement, but when the application needs to handle tens of thousands of concurrent connections, the scalability of this architecture will become a problem.

How does NGINX work?

How Does NGINX Work?

NGINX uses a predictable process model to schedule available hardware resources:

1. The main process performs privileged operations such as reading configuration and binding ports, and is also responsible for creating child processes (the three types below).

2. The cache loader process runs at startup, loads the disk-based cache into memory, and then exits. It is scheduled carefully so that its resource requirements are low.

3. The cache manager process runs periodically and prunes entries from the disk caches to keep them within the configured range.

4. Worker processes are the processes that perform all the actual tasks: handling network connections, reading and writing content to disk, communicating with upstream servers, etc.

In most cases, NGINX recommends running one worker process per CPU core to make the most efficient use of hardware resources. You can set the following directives in your configuration:

worker_processes auto

When the NGINX server is running, only the worker processes are busy. Each worker process handles multiple connections in a non-blocking manner to reduce context switching overhead.

Each worker process is single-threaded and runs independently, grabbing and handling new connections. Processes share cache data, session persistence data, and other shared resources through shared memory.

Worker Processes within NGINX

Inside the NGINX Worker Process

Each NGINX worker process is initialized with the NGINX configuration and is assigned a set of listen sockets by the master process.

NGINX worker processes listen to events on sockets (accept_mutex and kernel socket sharding) to decide when to start work. Events are initiated by new connections. These connections are assigned to state machines—the HTTP state machine is the most commonly used, but NGINX also implements state machines for streaming (native TCP) and a number of mail protocols (SMTP, IMAP, and POP3).

The state machine is essentially a set of instructions that tells NGINX how to handle a request. Most web servers that perform the same function as NGINX use a similar state machine—the implementation just differs.

Scheduling state machine

Scheduling the State Machine

Think of a state machine like the rules of chess. Every HTTP transaction is a chess game. On one side of the chessboard sits the web server - a grandmaster chess player who can make decisions quickly. On the other side is the remote client - a web browser accessing a site or application over a relatively slow network.

However, the rules of the competition can be complicated. For example, a web server might need to communicate with other parties (proxying for an upstream application), or communicate with an authentication server. Third-party modules of the web server can also extend the rules of the game.

Blocking state machine

A Blocking State Machine

Recall our previous description of processes and threads: a set of independent instruction sets that can be scheduled by the operating system and run on the CPU core. Most web servers and web applications use a one connection/one process or one connection/one thread model to play this chess game. Each process or thread contains an instruction to play the game to the end. During this process, the process is run by the server, and most of its time is spent "blocked", waiting for the client to complete its next action.

1. The web server process listens for new connections (new matches initiated by the client) on the listening socket.

2. After a new game is initiated, the process starts working. After each move, it enters a blocked state, waiting for the client to make the next move.

3. Once the game is over, the web server process will see if the client wants to start a new game (this is equivalent to a keepalive connection). If the connection is closed (the client leaves or times out), the web server process goes back to the listening state, waiting for a new match.

An important point to remember is that each active HTTP connection (each chess game) requires a dedicated process or thread (a grandmaster player). This architecture is very easy to extend with third-party modules ("new rules"). However, there is a huge imbalance here: a lightweight HTTP connection, represented by a file descriptor and a small amount of memory, maps to a single process or thread - which are very heavyweight operating system objects. This is convenient programming-wise, but it creates enormous waste.

NGINX is the real master

NGINX is a True Grandmaster

Perhaps you've heard of the tournament, in which a chess grandmaster takes on dozens of opponents at the same time.

Kiril Georgiev played against 360 players simultaneously in Sofia, the capital of Bulgaria, and eventually achieved a record of 284 wins, 70 draws and 6 losses.

This is how NGINX workers play “chess”. Each worker process is a grandmaster (remember: typically, each worker process occupies one CPU core) and can play against hundreds of players (actually thousands) simultaneously.

1. The worker process waits for events on the listening socket and the connecting socket.

2. Events occur on the socket and the worker process handles these events.

●Events on the listening socket mean: the client has started a new game. The worker process creates a new connection socket.

●An event on the connection socket means: the client moved a chess piece. The worker process responds quickly.

The worker process never stops on the network, it is always waiting for its "opponent" (client) to respond. When it has moved the pieces for this game, it will immediately go to deal with the next game, or take on a new opponent.

Why is it faster than a blocking multi-process architecture?

Why Is This Faster than a Blocking, Multi-Process Architecture?

NGINX scales well to support tens of thousands of connections per worker process. Each new connection creates another file descriptor and consumes a small amount of additional memory in the worker process. The additional cost of each connection is very small. The NGINX process can maintain a fixed CPU usage. There are also fewer context switches when there is no work to do.

In the blocking, one-connection/one-process model, each connection requires a lot of additional resources and overhead, and context switches (from one process to another) are very frequent.

If you want to learn more, check out this article on NGINX architecture written by Andrew Alexeev, VP of Corporate Development and co-founder of NGINX.

With proper system tuning, NGINX can scale to handle hundreds of thousands of concurrent HTTP connections per worker process without losing any information (new races) during traffic spikes.

Configuration updates and NGINX upgrades

Updating Configuration and Upgrading NGINX

The NGINX process architecture, which consists of only a small number of worker processes, makes updates to the configuration and even the binaries themselves very efficient.

Updating the NGINX configuration is a very simple, lightweight, and reliable operation. Just run nginx's reload command, which will check the configuration on disk and send a SIGHUP signal to the main process.

When the main process receives the SIGHUP signal, it does two things:

1. Reload the configuration and fork a new set of working processes. These new worker processes will immediately begin accepting connections and processing traffic (using the new configuration).

2. Send a signal to notify the old worker process to exit quietly. These old processes will no longer accept new connections. As soon as the HTTP request they are handling is finished, they close the connection cleanly. Once all connections are closed, the worker process exits.

This process will cause a small spike in CPU usage and memory usage, but this small spike is negligible compared to loading resources from active connections. You can reload the configuration multiple times per second. In rare cases, there will be problems while generations of workers wait for connections to close, but even if problems do occur, they will be resolved immediately.

The binary upgrade process for NGINX is even more amazing—you can upgrade NGINX itself on the fly without any dropped connections, downtime, or service interruptions to the server.

The binary upgrade process is similar to configuration updates. The new NGINX master process runs in parallel with the original master process, and they share the listening socket. Both processes are active, and their respective worker processes handle their own traffic. You can then notify the old master process and its workers to exit gracefully.

The entire process is described in more detail in Controlling NGINX.

in conclusion

Conclusion

This diagram of NGINX’s internals provides a high-level overview of how NGINX works, but behind this simple explanation are over a decade of innovation and optimization. These innovations and optimizations enable NGINX to perform well on a variety of hardware while also providing the security and reliability required by modern web applications.

The above is the full content of this article. I hope it will be helpful for everyone’s study. I also hope that everyone will support 123WORDPRESS.COM.