Detailed explanation of overlay network in Docker

Translated from Docker official documentation, original text: https://docs.docker.com/network/overlay/

Overlay networking creates a distributed network between multiple hosts where Docker daemons are located. This network overrides the host-specific network and allows containers to connect to it (including containers in cluster services) to communicate securely. Obviously, docker takes care of routing datagrams between the docker daemon source and destination containers.

When you initialize a swarm or join a Docker host to an existing swarm, two new networks are created on the host:

An overlay network called ingress is used to handle control and data transmission related to cluster services. When you create a cluster service and do not connect it to a user-defined overlay network, it is connected to the ingress network by default.

A bridge network called docker_gwbridge. Used to connect this Docker daemon to other daemons in the cluster.

You can create user-defined overlay networks with the docker network create command, just like you create user-defined bridge networks. Services and containers can be connected to multiple networks simultaneously. Services and containers can only communicate with other objects in the network where they are located.

Although both cluster services and individual containers can connect to the overlay network, the default behavior and configuration of the two are different. Therefore, the rest of this topic is divided into three parts: those applicable to all overlay networks; those applicable to networks in cluster services; and those applicable to overlay networks used by individual containers.

Applicable to all overlay network operations

Create an overlay network

✅Prerequisites

Firewall rules required for the Docker daemon using overlay networking

To allow Docker hosts in an overlay network to communicate with each other, you need to open the following ports:

1. TCP port 2377, used for cluster management related communications

2. TCP and UDP port 7946, used for communication between nodes

3. UDP port 4789 is used for data transmission on the overlay network

Before you can create an overlay network, you must either initialize your Docker daemon as a swarm manager using docker swarm init or join it to an existing swarm using docker swarm join.

Regardless of the method, an overlay network called ingress is created and used by default. Do this even if you don't plan to use cluster services.

Later you can create user-defined overlay networks.

To create an overlay network for use with cluster services, use the command shown below:

$ docker network create -d overlay my-overlay

To create a network that can be used both for swarm services and for individual containers to communicate with other individual containers in other Docker daemons, add the --attachable flag:

$ docker network create -d overlay --attachable my-attachable-overlay

You can specify IP address range, subnet, gateway, and other options. See docker network create --help for details.

Encrypted transmission on overlay network

All service management related transmissions are encrypted using the AES algorithm in GCM mode by default. The management nodes in the cluster rotate the encryption key every 12 hours.

If you want to encrypt application data, add --opt encrypted when creating the network. This parameter supports IPSEC encryption at the vxlan level. This operation will cause non-negligible performance degradation, so it should be tested before applying it to a production environment.

When you enable overlay encryption, Docker creates IPSEC tunnels on all nodes in the network to which the service is scheduled. These tunnels are also encrypted using the AES algorithm in GCM mode, and the encryption key is automatically rotated every 12 hours.

❌ Do not add Windows nodes to an overlay network that uses encrypted communications.

Encrypted communications on overlay networks are not supported on Windows. If a Windows node attempts to connect to an overlay network that uses encrypted communications, no error will be reported, but the node will not be able to communicate with other nodes.

Cluster-mode overlay network and separate containers

You can use the overlay network feature by using --opt encrypted --attachable or by adding unmanaged containers to the network.

$ docker network create --opt encrypted --driver overlay --attachable my-attachable-multi-host-network

Modify the default ingress network

Most users do not need to configure ingress networking. But Docker 17.05 and later versions allow you to do this. This feature is useful if the automatically selected subnet conflicts with an existing network on your network, or if you need to change other low-level network settings such as MTU.

To modify the ingress network, you need to delete and then create it. This requires you to complete the modification before creating the service in the cluster. If there are services that publish ports, delete them before you delete the ingress network.

When the ingress network does not exist, existing services without published ports can continue to provide services, but there is no load balancing function. Services that publish ports, such as WordPress services that publish port 80, will be affected.

Use docker network inspect ingress to check the ingress network, and then delete all services in the container connected to the ingress. These services are services that publish ports, such as the WordPress service that publishes port 80. If all of these services are not stopped, the next step will fail.

Delete the ingress network.

$ docker network rm ingress

WARNING! Before removing the routing-mesh network, make sure all the nodes in your swarm run the same docker engine version. Otherwise, removal may not be effective and functionality of newly created ingress networks will be impaired.

Are you sure you want to continue? [y/N]

3. Create a new overlay network with the ingress tag and add the configuration you want. The following example configures the MTU to 1200, sets the subnet to 10.11.0.0/16, and sets the gateway to 10.11.0.2.

$ docker network create \
 --driver-overlay \
 --ingress \
 --subnet=10.11.0.0/16 \
 --gateway=10.11.0.2 \
 --opt com.docker.network.mtu=1200 \
 my-ingress

Note: You can name the ingerss network anything else, but there can only be one. If you try to create a second one, it will fail.

4. Restart the service you stopped in the first step.

Modify the docker_gwbridge interface

docker_gwbridge is a virtual bridge that connects the overlay network (including the ingress network) to the physical network of a specific docker daemon. Docker automatically creates it when you initialize a cluster or join a docker host to a cluster, but it is not a docker device. It exists in the kernel of the Docker host. If you want to modify its configuration, you must do so before adding the host to the cluster, or temporarily remove the host from the cluster.

Stop Docker

Delete the docker_gwbridge interface

$ sudo ip link set docker_gwbridge down

$ sudo ip link del name docker_gwbridge

3. Start Docker, do not join or initialize the cluster

4. Use the docker network create command to manually create or re-create the docker_gwbridge bridge, adding your customized settings. The following examples use the 10.11.0.0/16 subnet.

$ docker network create \
--subnet 10.11.0.0/16 \
--opt com.docker.network.bridge.name=docker_gwbridge \
--opt com.docker.network.bridge.enable_icc=false \
--opt com.docker.network.bridge.enable_ip_masquerade=true \
docker_gwbridge

5. Initialize or join a cluster. Since the bridge already exists, Docker will not create it again with the default configuration.

Operations in cluster services

Publish ports on the overlay network

Cluster services connected to the same overlay network will expose all ports to each other. If a port is to be accessible from outside the service, it must be published using -p or --publish in docker service create or docker service update.

Both the legacy colon-separated syntax and the new comma-separated syntax are supported.

The longer syntax is better because it is somewhat self-explanatory.

Bypassing the routing network for cluster services

By default, cluster services publish ports through the routing network. When you connect to a published port on any cluster node (regardless of whether it is running the service represented by the port or not), you can be redirected to the node running the specified service. Docker effectively acts as a load balancer for your cluster of services. Services using the routing network run in virtual IP mode (VIP). Even if a service is run on a single node (via the --global flag), the routing mesh is used. When using a routing mesh, there is no guarantee which node handles a client's request.

To bypass the routing mesh, you can start the service in DNS Round Robin (DNSRR) mode. By setting the --endpoint-mode flag to dnsrr. You must run your own load balancer in front of the service. A DNS query for a service name on the Docker host returns a set of IP addresses of nodes running the specified service. Configure your load balancer to use this list and balance traffic between the nodes.

Separating control flow and data flow

By default, control traffic is communicated to the cluster manager and transmitted between applications running on the same network, although the control traffic is encrypted. You can configure Docker to use different network interfaces to handle different streams. When you initialize or join a cluster, specify --advertise-addr and --datapath-addr, respectively. You must do this on each node that you want to join the cluster.

Operations available to individual containers on the overlay network

Connecting standalone containers to the overlay network

The ingress network was created without the --attachable flag, which means that only cluster services can use it, not standalone containers. You can connect standalone containers to user-defined overlay networks that were created with the --attachablebiaojid option specified. This gives independent containers running on different Docker hosts the ability to communicate without having to set up routing on a specific Docker host.

Publish Ports

The above detailed explanation of the overlay network in Docker is all I have to share with you. I hope it can give you a reference. I also hope that you will support 123WORDPRESS.COM.