Looking at Tomcat's thread model from the connector component - BIO mode (recommended)

In higher versions of Tomcat, the default mode is to use NIO mode. In Tomcat 9, the implementation of BIO mode Http11Protocol has even been deleted. But understanding the working mechanism of BIO and its advantages and disadvantages is helpful for learning other modes. Only after comparison can you know the advantages of other models.

Http11Protocol represents the blocking HTTP protocol communication, which includes the entire process of receiving, processing, and responding to the client from the socket connection. It mainly contains JIoEndpoint component and Http11Processor component. At startup, the JIoEndpoint component will start listening on a certain port. When a request arrives, it will be thrown into the thread pool, which will process the task. During the processing, the HTTP protocol will be parsed by the protocol parser Http11Processor component, and matched to the specified container through the adapter for processing and response to the client.

Here we combine the Tomcat embedded in Spring Boot to see how the connector works. It is recommended to use a lower version of Spring Boot. In higher versions of Spring Boot, Tomcat 9 is already used. Tomcat 9 has removed the BIO implementation. The Spring Boot version I chose here is 2.0.0.RELEASE.

How to view the source code of the Connector component

We are now going to start analyzing the working process of the connector component through the source code of the Connector component. But there is so much source code for Tomcat, how should we read it? The previous article summarizes the Tomcat startup process, as shown in the following figure:

The above sequence diagram provides us with ideas for analyzing the source code of the Connector component: starting from the init method and start method of the connector component.

Connector component working sequence diagram

The Tomcat embedded in Spring Boot uses the NIO mode by default. If you want to study the BIO mode, you have to work on it yourself. Spring Boot provides the WebServerFactoryCustomizer interface, which we can implement to customize the Servlet container factory. The following is a configuration class I implemented myself. It simply sets the IO model to BIO mode. If you need to make other configurations, you can also make additional configurations in it.

@Configuration
public class TomcatConfig {

 @Bean
 public WebServerFactoryCustomizer tomcatCustomizer() {
  return new TomcatCustomerConfig();
 }

 public class TomcatCustomerConfig implements WebServerFactoryCustomizer<TomcatServletWebServerFactory> {
  @Override
  public void customize(TomcatServletWebServerFactory factory) {
   if (factory != null) {
    factory.setProtocol("org.apache.coyote.http11.Http11Protocol");
   }
  }
 }
}

After the above configuration, Tomcat's connector component will process requests in BIO mode.

Since Tomcat has a lot of code, it is not realistic to analyze all the code in one article. Here, I have sorted out the timing diagram of the connector component. Based on this timing diagram, I have analyzed several key code points. For other details, you can look at the code yourself according to my timing diagram. This code is not very complicated.

The key code here is in the init() method and start() method of JIoEndpoint. The init() method of JIoEndpoint mainly performs port binding of ServerSocket. The specific code is as follows:

@Override
public void bind() throws Exception {

 // Initialize thread count defaults for acceptor
 if (acceptorThreadCount == 0) {
  acceptorThreadCount = 1;
 }
 // Initialize maxConnections
 if (getMaxConnections() == 0) {
  // User hasn't set a value - use the default
  setMaxConnections(getMaxThreadsWithExecutor());
 }

 if (serverSocketFactory == null) {
  if (isSSLEnabled()) {
   serverSocketFactory =
    handler.getSslImplementation().getServerSocketFactory(this);
  } else {
   serverSocketFactory = new DefaultServerSocketFactory(this);
  }
 }
 //Here is the port binding for ServerSocket if (serverSocket == null) {
  try {
   if (getAddress() == null) {
    //If no specific address is specified, Tomcat will listen for requests from all addresses serverSocket = serverSocketFactory.createSocket(getPort(),
                getBacklog());
   } else {
    //Specify the specific address, Tomcat only listens for requests from this address serverSocket = serverSocketFactory.createSocket(getPort(),
                getBacklog(), getAddress());
   }
  } catch (BindException orig) {
   String msg;
   if (getAddress() == null)
    msg = orig.getMessage() + " <null>:" + getPort();
   else
    msg = orig.getMessage() + " " +
    getAddress().toString() + ":" + getPort();
   BindException be = new BindException(msg);
   be.initCause(orig);
   throw be;
  }
 }

}

Let's look at the start method of JIoEndpoint.

public void startInternal() throws Exception {

 if (!running) {
  running = true;
  paused = false;

  //Create thread pool if (getExecutor() == null) {
   createExecutor();
  }
  //Create ConnectionLatch
  initializeConnectionLatch();
  //Create the accept thread, which is the initial thread for request processing startAcceptorThreads();
  // Start async timeout thread
  Thread timeoutThread = new Thread(new AsyncTimeout(),
           getName() + "-AsyncTimeout");
  timeoutThread.setPriority(threadPriority);
  timeoutThread.setDaemon(true);
  timeoutThread.start();
 }
}

In the above code, we need to focus on the startAcceptorThreads() method. Let's take a look at the specific implementation of this Accept thread.

protected final void startAcceptorThreads() {
 int count = getAcceptorThreadCount();
 acceptors = new Acceptor[count];
 //According to the configuration, set a certain number of accept threads for (int i = 0; i < count; i++) {
  acceptors[i] = createAcceptor();
  String threadName = getName() + "-Acceptor-" + i;
  acceptors[i].setThreadName(threadName);
  Thread t = new Thread(acceptors[i], threadName);
  t.setPriority(getAcceptorThreadPriority());
  t.setDaemon(getDaemon());
  t.start();
 }
}

For the specific processing implementation of the Acceptor thread, focus on the run method.

protected class Acceptor extends AbstractEndpoint.Acceptor {

  @Override
  public void run() {

   int errorDelay = 0;
   // Loop until we receive a shutdown command
   while (running) {
    // Loop if endpoint is paused
    while (paused && running) {
     state = AcceptorState.PAUSED;
     try {
      Thread.sleep(50);
     } catch (InterruptedException e) {
      // Ignore
     }
    }

    if (!running) {
     break;
    }
    state = AcceptorState.RUNNING;

    try {
     //if we have reached max connections, wait
     //When the connection limit is reached, the acceptor thread enters a waiting state until other threads are released. This is a simple means of flow control by the number of connections. //This is achieved by implementing the AQS component (LimitLatch). The idea is to initialize the maximum limit of the synchronizer first, then increase the count variable by 1 for each socket received, and decrease the count variable by 1 for each socket closed.
     countUpOrAwaitConnection();
     Socket socket = null;
     try {
      //Accept the next socket connection. If no connection is made, this method will block socket = serverSocketFactory.acceptSocket(serverSocket);
     } catch (IOException ioe) {
      //If there is an exception, release a connection countDownConnection();
      errorDelay = handleExceptionWithDelay(errorDelay);
      throw ioe;
     }
     // Successful accept, reset the error delay
     errorDelay = 0;
     //Configure the socket appropriately if (running && !paused && setSocketOptions(socket)) {
      // Process this socket request, which is also the key point.
      if (!processSocket(socket)) {
       countDownConnection();
       // Close socket right away
       closeSocket(socket);
      }
     } else {
      countDownConnection();
      // Close socket right away
      closeSocket(socket);
     }
    } catch (IOException x) {
     if (running) {
      log.error(sm.getString("endpoint.accept.fail"), x);
     }
    } catch (NullPointerException npe) {
     if (running) {
      log.error(sm.getString("endpoint.accept.fail"), npe);
     }
    } catch (Throwable t) {
     ExceptionUtils.handleThrowable(t);
     log.error(sm.getString("endpoint.accept.fail"), t);
    }
   }
   state = AcceptorState.ENDED;
  }
 }

The processSocket(socket) in the thread processing class above is the method for processing specific requests. This method packages the request and "throws" it into the thread pool for processing. However, this is not the focus of the connector component. Later, we will introduce how Tomcat handles requests when introducing the request flow.

Here, we have briefly introduced Tomcat's BIO mode. In fact, you can see that if the BIO mode is simplified, it is the operation of the traditional ServerSocket, and the processing of requests is optimized with thread pool.

BIO Mode Summary

A brief description of each component in the above figure is given below.

Current limiting component LimitLatch

The LimitLatch component is a flow control component, which aims to prevent the Tomcat component from being overwhelmed by large flows. LimitLatch is implemented through the AQS mechanism. When this component starts, it first initializes the maximum limit value of the synchronizer, then increases the count variable by 1 for each socket received, and decreases the count variable by 1 for each socket closed. When the number of connections reaches the maximum value, the Acceptor thread enters a waiting state and no longer accepts new socket connections.

It should be noted that when the maximum number of connections is reached (the LimitLatch component has reached its maximum value and the acceptor component is blocked), the underlying operating system will continue to receive client connections and put the requests into a queue (backlog queue). This queue has a default length, the default value is 100. Of course, this value can be configured through the acceptCount attribute of the Connector node in server.xml. If a large number of requests come in within a short period of time and the backlog queue is full, the operating system will refuse to accept subsequent connections and return "connection refused".

In BIO mode, the maximum number of connections supported by the LimitLatch component is set through the maxConnections attribute of the Connector node in server.xml. If it is set to -1, it means no limit.

Acceptor

The responsibility of this component is very simple, which is to receive the Socket connection, make corresponding settings for the Socket, and then directly pass it to the thread pool for processing. The number of accept threads can also be configured.

Socket factory ServerSocketFactory

The Acceptor thread is obtained through the ServerSocketFactory component when accepting a socket connection. There are two ServerSocketFactory implementations in Tomcat: DefaultServerSocketFactory and JSSESocketFactory. Corresponding to the cases of HTTP and HTTPS respectively.

There is a variable SSLEnabled in Tomcat that is used to identify whether to use an encrypted channel. By defining this variable, you can decide which factory class to use. Tomcat provides an external configuration file for users to customize. In the following configuration, SSLEnabled="true" means using encryption, that is, using JSSESocketFactory to accept specific socket connections.

<Connector port="8443" protocol="org.apache.coyote.http11.Http11NioProtocol"
   maxThreads="150" SSLEnabled="true">
 <SSLHostConfig>
  <Certificate certificateKeystoreFile="conf/localhost-rsa.jks"
      type="RSA" />
 </SSLHostConfig>
</Connector>

Thread pool component

The thread pool in Tomcat is a simple modification of the thread pool in JDK. There is a slight difference in the thread creation strategy: the thread pool in Tomcat will not submit threads to the queue immediately after the number of threads is greater than coreSize, but will first determine whether the number of active threads has reached maxSize, and only submit threads to the queue after reaching maxSize.

The Executor of the Connector component is divided into two types: shared Executor and private Executor. A shared Executor is the Executor defined in the Service component.

Task definer SocketProcessor

Before throwing the Socket into the thread pool, we need to define how the task handles the Socket. SocketProcessor is the task definition, and this class implements the Runnable interface.

protected class SocketProcessor implements Runnable {
 //When debugging, you can start debugging from the run method of this class @Override
 public void run() { 
 	//Process the socket and output the response //Decrement the connection limiter LimitLatch by one //Close the socket}
}

The tasks of SocketProcessor are mainly divided into three parts: processing sockets and responding to clients, reducing the connection counter by 1, and closing the socket. Among them, the processing of sockets is the most important and the most complex. It includes reading the underlying socket byte stream, parsing the HTTP protocol request message (parsing of request line, request header, request body and other information), finding the Web project resources on the corresponding virtual host according to the path obtained from the request line parsing, and assembling the HTTP protocol response message according to the processing results and outputting it to the client.

We will not analyze the specific processing flow of the socket here for the time being, because this article is mainly about the thread model of the connector, which involves too many things and is easy to confuse. An article will be written later to analyze Tomcat's specific processing of the socket.

Summarize

This concludes this article about Tomcat's thread model from the perspective of connector components - the BIO model. For more information about Tomcat's thread model, please search 123WORDPRESS.COM's previous articles or continue to browse the following related articles. I hope you will support 123WORDPRESS.COM in the future!