Implementation of Docker cross-host network (manual)

1. Introduction to Macvlan

Before the emergence of Macvlan, we could only add multiple IP addresses to an Ethernet card, but not multiple MAC addresses, because the MAC address identifies an Ethernet card through its global uniqueness. Even if you use the method of creating ethx:y, you will find that the MAC addresses of all these "network cards" are the same as ethx. In essence, they are still a network card, which will limit your ability to perform many Layer 2 operations. With Macvlan technology, you can do this.

Macvlan allows you to configure multiple virtual network interfaces on a network interface of the host. These network interfaces have their own independent MAC addresses and can also be configured with IP addresses for communication. The virtual machines or container networks under Macvlan are in the same network segment as the host and share the same broadcast domain. Macvlan is similar to Bridge, but because it omits the existence of Bridge, it is simpler to configure and debug, and its efficiency is relatively high. In addition, Macvlan itself also perfectly supports VLAN.

Data transmission between the same VLAN is achieved through Layer 2 mutual access, that is, MAC address, without the need for routing. By default, users in different VLANs cannot communicate directly. If they want to communicate, they need a layer-3 device for routing. The same is true for Macvlan. The virtual network card virtualized by Macvlan technology is logically equivalent to the physical network card. The physical network card is equivalent to a switch, which records the corresponding virtual network card and MAC address. When the physical network card receives a data packet, it will determine which virtual network card the packet belongs to based on the destination MAC address. This means that as long as the data packet is sent from the Macvlan sub-interface (or sent to the Macvlan sub-interface), the physical network card only receives the data packet but does not process it. So this leads to a problem: the IP on the local Macvlan network card cannot communicate with the IP on the physical network card! We will discuss the solution to this problem in the next section.

Simply put, the Macvlan virtual network card device is parasitic on the physical network card device. When sending a packet, it calls its own packet sending function, finds the parasitic physical device, and then sends the packet through the physical device. When receiving packets, the data packets are processed by registering the rx_handler callback function of the parasitic physical device.

2. Briefly introduce the manual process

As its name suggests, macvlan is a network card virtualization technology that can virtualize multiple interfaces from a physical network card. Each interface can be configured with a MAC address and its own IP address. Each interface is like a switch port and can be divided into VLANs.

What macvlan does is actually connect these virtual interfaces directly to the Docker container to achieve the purpose of communication. One macvlan network corresponds to one interface, and different macvlan networks are assigned different subnets. Therefore, the same macvlan networks can communicate with each other, but different macvlan networks cannot communicate at the second layer and need the help of a third-layer router to complete the communication. The following shows the communication process between two different macvlan networks.

We use a Linux host, configure its routing table and iptables, and make it a router (virtual, of course), so that we can complete the data exchange between different macvlan networks. Of course, there is nothing wrong with using a physical router.

3. Features of Macvlan:

1. Allow users to set multiple MAC addresses on the same physical network card.
2. As mentioned above, the network card with the MAC address set above is called a sub-interface, and the physical network card is called a parent interface.
3. The parent interface can be a physical interface (eth0), an 802.1q sub-interface (eth0.10), or a bonding interface.
4. Not only the MAC address can be set on the parent/sub interface, but the IP address can also be set.
5. The sub interface cannot communicate directly with the parent interface (VMs or containers with sub interfaces cannot communicate directly with the host).
As mentioned above, if the VM or container needs to communicate with the host, an additional sub 6.interface must be created for the host to use.
7. Sub interfaces are usually named in the form of mac0@eth0 for easy identification.
Here is a picture to explain what it looks like after setting up Macvlan:

4. Experimental Environment

Turn off firewall and disable selinux, change hostname

[root@localhost ~]# hostnamectl set-hostname docker01
[root@localhost ~]# su -
Last login: Feb 17 08:20:36 CST 2019 from 192.168.1.1pts/0 [root@docker01 ~]# systemctl stop firealld
Failed to stop firealld.service: Unit firealld.service not loaded.
[root@docker01 ~]# setenforce 0
setenforce: SELinux is disabled
[root@docker01 ~]# systemctl daemon-reload 
[root@docker01 ~]# systemctl restart docker
4.1 Single network communication of macvlan

4.1 Single network communication of macvlan

1) Enable the promiscuous mode of the network card

//This operation needs to be performed on both docker01 and docker02_.

[root@docker01 ~]# ip link show ens33
// Check the network card mode 

[root@docker01 ~]# ip link set ens33 promisc on
//Create a network card in promiscuous mode [root@docker01 ~]# ip link show ens33
// Check the network card mode 

2) Create a macvlan network on docker01.

[root@docker01 ~]# docker network create -d macvlan --subnet 172.22.16.0/24 --gateway 172.22.16.1 -o parent=ens33 mac_net1
// Create a macvlan mode network -o parent=which network card is bound to [root@docker01 ~]# docker network ls
// Check the network card information 

3) Run a container based on the created macvlan network

[root@docker01 ~]# docker run -itd --name bbox1 --ip 172.22.16.10 --network mac_net1 busybox

4) Create a macvlan network on docker02. (It should be exactly the same as the macvlan of docker01)

[root@docker02 ~]# docker network create -d macvlan --subnet 172.22.16.0/24 --gateway 172.22.16.1 -o parent=ens33 mac_net1

[root@docker02 ~]# docker network ls 

5) On docker02., run a container based on the created macvlan network and verify the communication with the container on docker01.

[root@docker02 ~]# docker run -itd --name bbox2 --network mac_net1 --ip 172.22.16.20 busybox
//Create a container based on busybox[root@docker02 ~]# docker exec -it bbox2 /bin/sh
//Enter bbox2 container/ # ping 172.22.16.10
//Ping the host of docker01 

4.2MacVLAN multi-network communication

1) docker01 and docker02 verify the kernel module 8021q encapsulation

The problem that macvlan needs to solve: Produce a new virtual network card based on the real ens33 network card.

[root@docker01 ~]# modinfo 8021q
//Verify kernel module 8021q encapsulation 

[root@docker01 ~]# modprobe 8021q
//If the kernel module is not enabled, run the above command to import it

2) docker01 creates a virtual network card based on ens33

Modify the ens33 network card configuration file

[root@docker01 ~]# cd /etc/sysconfig/network-scripts/
[root@docker01 network-scripts]# vim ifcfg-ens33 

Manually add a virtual network card configuration file

[root@docker01 ~]# cd /etc/sysconfig/network-scripts/
[root@docker01 network-scripts]# cp -p ifcfg-ens33 ifcfg-ens33.10
//-p retains the properties of the source file or directory [root@docker01 network-scripts]# vim ifcfg-ens33.10
//Modify ens33.10 network card configuration file BOOTPROTO=none
NAME=ens33.10
DEVICE=ens33.10
ONBOOT=yes
IPADDR=192.168.10.10
PREFIX=24
GATEWAY=192.168.10.2
VLAN=yes

Note that the IP must be distinguished from the ens33 network segment to ensure the consistency of the gateway and network segment IP, the consistency of the device name and configuration file, and enable the VLAN support mode.

Create a second virtual NIC configuration file

[root@docker01 network-scripts]# cp -p ifcfg-ens33.10 ifcfg-ens33.20
[root@docker01 network-scripts]# vim ifcfg-ens33.20
//Modify ens33.20 network card configuration file BOOTPROTO=none
NAME=ens33.20
DEVICE=ens33.20
ONBOOT=yes
IPADDR=192.168.20.20
PREFIX=24
GATEWAY=192.168.20.2
VLAN=yes

Operation on docker01, enable the created virtual network card:

[root@docker01 network-scripts]# ifup ifcfg-ens33.10 
[root@docker01 network-scripts]# ifup ifcfg-ens33.20
[root@docker01 network-scripts]# ifconfig
//Check IP 

3) docker02 creates a virtual network card based on ens33

Modify the ens33 network card configuration file

[root@docker02 ~]# cd /etc/sysconfig/network-scripts/
[root@docker02 network-scripts]# vim ifcfg-ens33 

Manually add a virtual network card configuration file

[root@docker02 ~]# cd /etc/sysconfig/network-scripts/
[root@docker02 network-scripts]# cp -p ifcfg-ens33 ifcfg-ens33.10
//-p retains the properties of the source file or directory [root@docker02 network-scripts]# vim ifcfg-ens33.10
//Modify ens33.10 network card configuration file BOOTPROTO=none
NAME=ens33.10
DEVICE=ens33.10
ONBOOT=yes
IPADDR=192.168.10.11
PREFIX=24
GATEWAY=192.168.10.2
VLAN=yes

Note that the IP must be distinguished from the ens33 network segment to ensure the consistency of the gateway and network segment IP, the consistency of the device name and configuration file, and enable the VLAN support mode.

Create a second virtual NIC configuration file

[root@docker02 network-scripts]# cp -p ifcfg-ens33.10 ifcfg-ens33.20
[root@docker02 network-scripts]# vim ifcfg-ens33.20
//Modify ens33.20 network card configuration file BOOTPROTO=none
NAME=ens33.20
DEVICE=ens33.20
ONBOOT=yes
IPADDR=192.168.20.21
PREFIX=24
GATEWAY=192.168.20.2
VLAN=yes

Operation on docker02, enable the created virtual network card:

[root@docker02 network-scripts]# systemctl restart network
[root@docker02 network-scripts]# ifup ifcfg-ens33.10 
[root@docker02 network-scripts]# ifup ifcfg-ens33.20
[root@docker02 network-scripts]# ifconfig
//Check IP 

4) docekr01 and docker02 create macvlan network based on virtual network card

[root@docker02 network-scripts]# docker network create -d macvlan --subnet 172.16.10.0/24 --gateway 172.16.10.1 -o parent=ens33.10 mac_net10
//Create a new network card based on ens33.10
[root@docker02 network-scripts]# docker network create -d macvlan --subnet 172.16.20.0/24 --gateway 172.16.20.1 -o parent=ens33.20 mac_net20
//Create a new network card based on ens33.20

5) Docker01 deploys a private warehouse
Docker01

docker pull registry
//Download the registry image docker run -itd --name registry -p 5000:5000 --restart=always registry:latest 
//Based on the registry image, start a container docker tag busybox:latest 192.168.1.11:5000/busybox:v1 
//Rename the container to a label docker ps 

vim /usr/lib/systemd/system/docker.service #13 line modification ExecStart=/usr/bin/dockerd --insecure-registry 192.168.1.11:5000 

systemctl daemon-reload 
systemctl restart docker.service 
//Restart docker
docker push 192.168.1.11:5000/busybox:v1
//Upload the container to the private repository docker images

Docker02

vim /usr/lib/systemd/system/docker.service #13 line modification ExecStart=/usr/bin/dockerd --insecure-registry 192.168.1.11:5000 

systemctl daemon-reload 
systemctl restart docker.service 
//Restart docker
docker pull 192.168.1.11/busybox:v1 
//Download the image just uploaded

6) docker01 and docker02 create containers based on busybox:v1 image and network cards mac_net10 and mac_net20.
Docker01

[root@docker01 ~]# docker run -itd --name bbox10 --network mac_net10 --ip 172.16.10.10 192.168.1.11:5000/busybox:v1
[root@docker01 ~]# docker run -itd --name bbox20 --network mac_net20 --ip 172.16.20.20 192.168.1.11:5000/busybox:v1
**Docker02**

[root@docker02 ~]# docker run -itd --name bbox10 --network mac_net10 --ip 172.16.10.10 192.168.1.11:5000/busybox:v1
[root@docker02 ~]# docker run -itd --name bbox20 --network mac_net20 --ip 172.16.20.20 192.168.1.11:5000/busybox:v1

***Just note that our operations here are exactly the same as those in docker01 and above. The order of operations is roughly:
Verify 8021q core encapsulation

Create new virtual network cards based on the ens33 network card, ens33.10 and ens33.20 (note that ens33.10 and ens33.20 on docker01 must be in the same network segment and the IP addresses cannot conflict) and run containers based on this network. (Note that the containers on docker01 are all based on the macvlan network just created, but the IP addresses cannot conflict)

7) Verification
Enter container bbox10 on docker01. and communicate with bbox11 on docker02.
Enter container bbox20 on docker01. and communicate with bbox21 on docker02.
Note: VMware's network must be set to Bridge mode.
Now set the network mode of docker01 and docker02 to bridge mode

Test whether the host with the same network card can ping

[root@docker01 ~]# docker exec -it bbox10 /bin/sh
/# ping 172.16.20.20 

[root@docker02 ~]# docker exec -it bbox20 /bin/sh
/# ping 172.16.20.20

5. Limitations of Macvlan

Macvlan is a nearly ideal solution for connecting VMs or containers to the physical network via Layer 2, but it also has some limitations:

1. The switch to which the Linux host is connected may limit the number of MAC addresses on the same physical port. While you could have your network administrator change these policies, sometimes that's just not feasible (like if you're doing a quick PoC demo for a client).
2. Many NICs also have a limit on the number of MAC addresses on the physical card. Exceeding this limit will affect system performance.
3.IEEE 802.11 doesn't like multiple MAC addresses on the same client, which means your Macvlan subinterfaces won't be able to communicate on either the wireless card or the AP. You can use complex methods to break through this limitation, but there is a simpler way, which is to use Ipvlan. If you are interested, you can check the relevant information yourself.

6. Summary

macvlan is a network card virtualization technology that can virtualize multiple network cards from one network card.

A specific communication mode of macvlan. The commonly used mode is bridge.

In Docker, macvlan only supports bridge mode.

The same macvlan can communicate, but different macvlans cannot communicate at the second layer. Communication can be achieved through layer 3 routing.

Consider this:

The similarities and differences between macvlan bridge and bridge There is also a similar technology where multiple virtual network cards share the same MAC address but have independent IP addresses. What is this technology?

The above is the full content of this article. I hope it will be helpful for everyone’s study. I also hope that everyone will support 123WORDPRESS.COM.