Docker deploys Macvlan to achieve cross-host network communication

Basic concepts:

Macvlan working principle:

Macvlan is a network interface supported by the Linux kernel. The required Linux builds are v3.9–3.19 and 4.0+; by creating Macvlan subinterfaces for the physical network card, a physical network card is allowed to have multiple independent MAC addresses and IP addresses. The virtualized sub-interface will be directly exposed to the adjacent physical network. From the outside, it looks like the network cable is divided into multiple strands and connected to different hosts respectively. After the physical network card receives the packet, it will determine whether the packet needs to be handed over to the virtual network card based on the destination MAC address of the received packet.

When the container needs to be directly connected to the physical network, Macvlan can be used. Macvlan itself does not create a network. In essence, it first makes the host physical network card work in 'promiscuous mode', so that the MAC address of the physical network card will be invalid, and the physical network card can receive all traffic in the layer 2 network. The next step is to create a virtual network card on this physical network card and assign a MAC address to the virtual network card to achieve multiple uses of one card. From the perspective of the physical network, each virtual network card is a separate interface.

When using Macvlan, you need to pay attention to the following points:

• The container is directly connected to the physical network, which is responsible for allocating IP addresses. This may result in the exhaustion of physical network IP addresses. Another consequence is network performance issues. As more hosts are connected to the physical network, the proportion of broadcast packets increases rapidly, causing network performance to degrade.
• A network on the host needs to work in 'promiscuous mode';
• As mentioned above, the MAC address of a physical network card working in promiscuous mode will become invalid. Therefore, containers running in this mode cannot communicate with the external network, but it will not affect the communication between the host and the external network.
• In the long run, bridge networks and overlay networks are better choices because virtual networks should be isolated from physical networks rather than shared.

Project environment:

Two docker hosts: (centos7)
docker01: 172.16.1.30
docker02: 172.16.1.31

Project Operation:

Example 1: macvlan cross-host single network solution:

docker01:

(1) Enable the promiscuous mode of the ens33 network card and enable multiple virtual interfaces of the network card.

[root@sqm-docker01 ~]# ip link set ens33 promisc on

##Check the status of the network card:
[root@sqm-docker01 ~]# ip link show ens33 

(2) Create a macvlan network:

[root@sqm-docker01 ~]# docker network create -d macvlan --subnet 172.16.100.0/24 --gateway 172.16.100.1 -o parent=ens33 mac_net1

Parameter explanation:
-o: Which network card to bind to (based on ens33 network card)

(3) Run a container based on the network just created:

[root@sqm-docker01 ~]# docker run -itd --name box1 --ip 172.16.100.10 --network mac_net1 busybox

docker02: (same operation as docker01)

Enable promiscuous mode [root@sqm-docker02 ~]# ip link set ens33 promisc on
[root@sqm-docker02 ~]# ip link show ens33 

//Create a macvlan network [root@sqm-docker02 ~]# docker network create -d macvlan --subnet 172.16.100.0/24 --gateway 172.16.100.1 -o parent=ens33 mac_net1

//Run a container:
[root@sqm-docker02 ~]# docker run -itd --name box2 --network mac_net1 --ip 172.16.100.20 busybox

(4) Test that two containers on two hosts communicate with each other:

Note:

The reason why the ping is successful is that both containers are based on real ens33 network cards, so the ens33 network cards on the host must be able to communicate with each other. This method can only ping the IP address, but cannot ping the container name.

Example 2: macvlan cross-host multi-network solution:

(1) First check the 8021q module of the host kernel:

[root@sqm-docker01 ~]# modinfo 8021q 

##If you do not see the module, you need to execute the following command to load it:
[root@sqm-docker01 ~]# modprobe 8021q

Enable routing forwarding:
[root@sqm-docker01 ~]# echo "net.ipv4.ip_forward = 1" > /etc/sysctl.conf 
[root@sqm-docker01 ~]# sysctl -p
net.ipv4.ip_forward = 1

(2) Modify network configuration information:

docker01:

[root@sqm-docker01 ~]# cd /etc/sysconfig/network-scripts/
[root@sqm-docker01 network-scripts]# ls 

[root@sqm-docker01 network-scripts]# vim ifcfg-ens33 

Create a subnet card based on the ens33 network card:

[root@sqm-docker01 network-scripts]# cp -p ifcfg-ens33 ifcfg-ens33.10 #Customize network card name [root@sqm-docker01 network-scripts]# cp -p ifcfg-ens33 ifcfg-ens33.20

-p: means retaining the original attributes (permissions)

//Modify ens33.10 network card:
[root@sqm-docker01 network-scripts]# vim ifcfg-ens33.10
##Only keep the following options: 

//Modify ens33.20 network card:
[root@sqm-docker01 network-scripts]# vim ifcfg-ens33.20 
The configuration is the same as ens33.10, only the IP address needs to be modified: 

(3) Start the subnet card:

[root@sqm-docker01 network-scripts]# ifup ifcfg-ens33.10 
[root@sqm-docker01 network-scripts]# ifup ifcfg-ens33.20

//View network information [root@sqm-docker01 network-scripts]# ifconfig 

(4) Create a macvlan network based on ens33.10 and ens33.20:

Note: Different network segments have different network names

[root@sqm-docker01 ~]# docker network create -d macvlan --subnet 172.16.200.0/24 --gateway 172.16.200.1 -o parent=ens33.10 mac_net10

[root@sqm-docker01 ~]# docker network create -d macvlan --subnet 172.16.210.0/24 --gateway 172.16.210.1 -o parent=ens33.20 mac_net20

(5) Run two containers based on the above network:

[root@sqm-docker01 ~]# docker run -itd --name test1 --ip 172.16.200.10 --network mac_net10 busybox

[root@sqm-docker01 ~]# docker run -itd --name test2 --ip 172.16.210.10 --network mac_net20 busybox

Deploy docker02:

The operation is basically the same as docker01. Note that the network segment is the same, but the host IP is different.

#The following operations will not be explained:

Enable routing forwarding:
[root@sqm-docker01 ~]# echo "net.ipv4.ip_forward = 1" > /etc/sysctl.conf 
[root@sqm-docker01 ~]# sysctl -p
net.ipv4.ip_forward = 1

[root@sqm-docker02 network-scripts]# pwd
/etc/sysconfig/network-scripts
[root@sqm-docker02 network-scripts]# vim ifcfg-ens33 

[root@sqm-docker02 network-scripts]# cp -p ifcfg-ens33 ifcfg-ens33.10
[root@sqm-docker02 network-scripts]# cp -p ifcfg-ens33 ifcfg-ens33.20

[root@sqm-docker02 network-scripts]# vim ifcfg-ens33.10 

[root@sqm-docker02 network-scripts]# vim ifcfg-ens33.20 

The above is the full content of this article. I hope it will be helpful for everyone’s study. I also hope that everyone will support 123WORDPRESS.COM.

 [root@sqm-docker02 network-scripts]# ifup ifcfg-ens33.10 [root@sqm-docker02 network-scripts]# ifup ifcfg-ens33.20

 //創建macvlan網絡：[root@sqm-docker02 ~]# docker network create -d macvlan --subnet 172.16.200.0/24 --gateway 172.16.200.1 -o parent=ens33.10 mac_net10[root@sqm-docker02 ~]# docker network create -d macvlan --subnet 172.16.210.0/24 --gateway 172.16.210.1 -o parent=ens33.20 mac_net20

 //運行容器（ip地址不同）：[root@sqm-docker02 ~]# docker run -itd --name test3 --network mac_net10 --ip 172.16.200.11 busybox[root@sqm-docker02 ~]# docker run -itd --name test4 --network mac_net20 --ip 172.16.210.11 busybox

//確保容器正常運行:

(6) Test that containers can communicate across hosts: (Note: If you are using VMware, you must change the default NAT mode of the two hosts to bridge mode to enable normal communication due to VMware virtual machines)
Test3 communicates with test1 (same network segment):

Test4 communicates with test2 (same network segment):

Troubleshooting ideas: If the hosts cannot communicate after deployment, first confirm whether the firewall or iptables rules are closed or released, whether selinux is disabled, and then check whether the ens33 network card configuration file and its subnet card content are modified incorrectly. Finally, check whether the network segment definition is incorrect when you create the macvlan network, or whether the IP address is incorrectly specified when running the container.

----------------------Macvlan multi-network cross-host communication deployment completed---------------------

Extended knowledge points:
Suppose we run a t1 container, and then the t2 container uses the network stack of the t1 container.

 [root@sqm-docker03 ~]# docker run -itd --name t1 busybox[root@sqm-docker03 ~]# docker exec t1 ip a

[root@sqm-docker03 ~]# docker run -it --name t2 --network container:t1 busybox

 //接下來在t1容器中操作：[root@sqm-docker03 ~]# docker exec -it t1 bin/sh

Then you can also see this service in the t2 container:

The above is the basic content of deploying the network stack. I don’t use it often. It is just to enable other containers to share the resources in one container.