Docker container from entry to obsession (recommended)

1. What is Docker?

Everyone knows about virtual machines. Installing a Linux virtual machine on Windows is a common solution for most programmers. Most of the company's production environments are also virtual machines. Virtual machines virtualize physical hardware resources and allocate and use them on demand. Virtual machines are used exactly like real operating systems. When they are no longer needed, resources can be reclaimed by simply deleting the virtual machine files, which is very convenient for centralized management.

Because virtual machines are very large and consume a lot of hardware resources, Linux has developed another virtualization technology, namely Linux Containers (abbreviated as LXC), which does not simulate a complete operating system like a virtual machine, but provides the same effect as a virtual machine. If virtual machines are isolation at the operating system level, then containers are isolation at the process level. It is conceivable that the advantages of this level of isolation are undoubtedly fast and resource-saving.

Docker is a package of Linux containers, providing a simple and practical user interface. It is currently the most popular Linux container solution.

Here is the definition from the encyclopedia:

Docker is an open source application container engine based on the Go language and complies with the Apache 2.0 protocol. Docker allows developers to package their applications and dependent packages into a portable container and then publish them to any popular Linux machine. It can also achieve virtualization. Containers use a complete sandbox mechanism and do not have any interfaces with each other.

2. What problems does docker solve?

1. Solve the problem of virtual machine resource consumption.

Virtual machines run on the server operating system, client operating systems run on the virtual machines, and user applications run on the client operating systems. 80% of a server's resource overhead is spent on hardware virtualization and the client operating system itself.

Figure 1. Differences between virtual machine architecture and container architecture

As shown in Figure 1, if Docker container technology is used, a virtual server runs on the container, and the user's application runs in the virtual server. The virtual server and the server operating system use the same kernel, and the file system of the virtual server uses the file system of the physical server, but is isolated. It seems that each virtual server has its own independent file system. A virtual bridge device is established on the physical server, and each virtual server is connected to the network through the virtual bridge device. The virtual server directly uses the CPU, memory, and hard disk of the physical server and does not virtualize the hardware. Therefore, there is no resource consumption occupied by hardware virtualization and client operating system. The performance of each virtual server is close to that of the physical server.

An ordinary home computer running a Linux virtual machine may be very slow, but you can use Docker to virtualize dozens or even hundreds of virtual Linux servers. If you switch to a powerful server, you can use Docker to provide private cloud services.

2. Rapid deployment.

The difficulty of software development lies in the environment configuration. The software that runs on your own computer may not run on another machine unless the operating system is set up correctly and the various components and libraries are installed correctly. For example, to deploy a web system developed in Java, the computer must have Java and the correct environment variables installed, and may also need to have tomcat and nginx installed. If you change the machine, you have to start all over again.

Using docker, you can package the application and its dependencies in one file (docker image file). Running this file will start a virtual server. Starting the application or service in the virtual server is just like running it on a real physical machine. With docker, you can deploy it once and run it everywhere. It can also be used for automated releases.

3. Provide a disposable environment.

For example, testing other people's software locally, providing a unit testing and building environment during continuous integration, and starting or shutting down a virtual server is as simple and fast as starting or shutting down a process.

4. Provide flexible cloud services.

Because Docker containers can be turned on and off at any time, they are very suitable for dynamic expansion and reduction.

5. Build a microservice architecture .

Through multiple containers, one machine can run many virtual servers, so a microservice architecture or a distributed architecture can be simulated on one machine.

3. Docker installation, deployment and use

This article introduces the installation and use of Ubuntu 18.04 system. For other operating systems, please refer to the official documentation at https://docs.docker.com/.

1. Install Docker Engine

Get the latest version of the Docker installation package

aaron@ubuntu:~$ wget -qO- https://get.docker.com/ | sh

Execute the above command and enter the current user password to automatically download the latest version of the docker installation package and install it automatically.

After the installation is complete, there is a prompt:

If you would like to use Docker as a non-root user, you should now consider
add your user to the "docker" group with something like:

 sudo usermod -aG docker aaron

Remember that you will have to log out and back in for this to take effect!

WARNING: Adding a user to the "docker" group will grant the ability to run
     containers which can be used to obtain root privileges on the
     docker host.
     Refer to https://docs.docker.com/engine/security/security/#docker-daemon-attack-surface
     for more information.

When you want to run docker directly as a non-root user, you need to execute

sudo usermod -aG docker aaron 

Run the command to add user aaron to the docker user group, and then log in again. Otherwise, the following error will be reported:

docker: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.38/containers/create: dial unix /var/run/docker.sock: connect: permission denied.
See 'docker run --help'.

Execute the following command to start the Docker engine:

aaron@ubuntu:~$ sudo service docker start

After successful installation, the system is set to start automatically at boot. If you want to set it manually, execute the following command:

sudo systemctl enable docker
sudo systemctl start docker

Test run

aaron@ubuntu:~$ sudo docker run hello-world

2. Using Docker

1. Understand the architecture of Docker

Before using, first understand the architecture of Docker, as shown in the following figure:

Docker architecture diagram

• A docker image is a file stored in the docker registry and is a template for creating a docker container.
• A Docker container is an application or a group of applications that run independently, which can be understood as the virtual server mentioned above.
• A Docker host is a physical or virtual machine that runs the Docker daemon and containers.
• The docker client uses the docker API to communicate with the docker daemon through the command line or other tools.

As users, we use the docker client directly.

2. Docker command

View the help information of the docker command

docker --help #help information for all docker commands docker COMMAND --help #help information for specific docker commands COMMAND

View Docker information

docker info

You can see the container pool, used data size, total data size, basic container size, number of currently running containers, etc.

Search for images and search for container images made by others on the Internet.

docker search ubuntu
docker search centos

ubuntu images

From here we can see that some images have integrated applications such as PHP, Java, and Ansible. We can also create image files containing our own applications or services and pass them to others. They can directly use Docker to open the container without any additional operations and without consuming resources like virtual machines. They can run your applications or services. Isn’t it very convenient? !

Download container images made by others from the Internet.

docker pull centos
docker pull ubuntu

Import the downloaded container image file

docker load < image_xxx.tar

View Mirror

docker images
docker images -a

Check the image

docker inspect ubuntu

You can see the basic information of the container image.

Delete the image, specify the deletion by the image id

docker rmi ubuntu

Delete all images

docker rmi $(docker images -q)

Show mirror history

docker history ubuntu

Running the container

A Docker container can be understood as a process running in a sandbox, which contains the resources necessary for the process to run, including the file system, system libraries, shell environment, etc. However, this sandbox will not run any programs by default. You need to run a process in the sandbox to start a container. This process is the only process of the container, so when the process ends, the container will also stop completely.

Run the ubuntu container and enter the interactive environment

aaron@ubuntu:~$ docker run -i --name="ubuntu1" --hostname="ubuntu1" ubuntu /bin/sh
cat /etc/hosts
127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
172.17.0.2 ubuntu1
whoami
root
uname -a
Linux ubuntu1 4.15.0-34-generic #37-Ubuntu SMP Mon Aug 27 15:21:48 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

In the above command, we created a container named ubuntu1 and set the host name of the container to ubuntu1. After entering the /bin/sh command, we printed the contents of the hosts file and checked the kernel version (which is consistent with the local operating system version). Various Linux commands can be used here, just like using commands in the new operating system. In the same way, we create a ubuntu2 container in a new terminal and use

docker ps

View the running containers.

View running containers

Type exit to exit the container.

docker run -d ubuntu

A long string of alphanumeric characters will appear, which is the container ID. Please note that the container must have a continuously running process, otherwise the container will automatically exit soon.
Run the container and specify the MAC address

docker run -d --name='centos3' --hostname='centos3' --mac-address="02:42:AC:11:00:24" docker-centos6.10-hadoop-spark

List all containers

docker ps -a

List the most recently started containers

docker ps -l

Check the container

docker inspect centos1

You can get information about the container.

Get the container CID

docker inspect -f '{{.Id}}' centos1

Get the container PID

docker inspect -f '{{.State.Pid}}' centos1

Get the container IP

docker inspect -f '{{.NetworkSettings.IPAddress}}' centos1

Get container gateway

docker inspect -f '{{.NetworkSettings.Gateway}}' centos1

Get container MAC

docker inspect -f '{{.NetworkSettings.MacAddress}}' centos1

View the container IP address

docker inspect -f '{{.NetworkSettings.IPAddress}}' centos1

Connecting Containers

ssh container's IP address

Enter password: 123456

After the container is running, you can enter the container in another way

docker exec -it centos /bin/sh

View the logs of the container running process

docker logs centos1

List the files or directories that have been changed in a container. The list will show three types of events: A added; D deleted; C changed

docker diff centos1

And the initial container image project, the user or system can view those directory files added/modified/deleted.
View the processes running in the container

docker top centos1

Copy files/directories in the container to the local server

docker cp centos1:/etc/passwd /tmp/
ls /tmp/passwd

The container files can also be copied to the server through the network IP address, which is more convenient.

Stop the container

docker stop centos1

Stop all containers

docker kill $(docker ps -a -q)

Start the container

docker start centos1

Deleting a single container

docker stop centos1
docker rm centos1

Before deleting a container, you must stop it.

Delete all containers

docker kill $(docker ps -a -q)
docker rm $(docker ps -a -q)

3. The concept of volume

In order to save (persist) data and share data between containers, Docker proposes the concept of volume. A volume is a specific directory of a container. The files under this directory are stored on the host machine instead of in the container's file system.

A data volume is a special directory that can be used by one or more containers. It bypasses the container's default file system and provides many useful features:
(1) Data volumes can be shared and reused between containers;
(2) Modifications to the data volume will take effect immediately;
(3) Updates to the data volume will not affect the image;
(4) By default, the data volume will always exist even if the container is deleted.

Note: The use of data volumes is similar to mounting a directory in Linux. The files in the directory designated as the mount point in the container will be hidden, and only the mounted data volume can be displayed.

Creating and using data volumes

mkdir -p /root/volume1
mkdir -p /root/volume2
docker run -d -v /volume1 --name='centos5' docker-centos6.10-hadoop-spark
docker run -d -v /root/volume1:/volume1 --name='centos6' docker-centos6.10-hadoop-spark
docker run -d -v /root/volume1:/volume1 -v /root/volume2:/volume2 --name='centos7' docker-centos6.10-hadoop-spark
docker run -d -v /root/volume1:/volume1:ro --name='centos8' docker-centos6.10-hadoop-spark

Use the docker run command to create a container, specify the -v flag to create a data volume and mount it in the container; you can mount multiple data volumes; you can set the volume's read-only attribute; you do not need to specify a directory for the server mapping, the system will automatically specify the directory, and you can view the mapped path through docker inspect.

Enter these containers respectively and view the /volume1 and /volume2 directories.

Data volume sharing

If you want to grant one container access to another container's data volumes, you can use the -volumes-from parameter to do so.

Data volume container

If there is some continuously updated data that needs to be shared between containers, it is best to create a data volume container.

A data volume container is actually a normal container that is specifically used to provide data volumes for other containers to mount.

(1) Create a data volume container named dbdata

docker run -d -v /dbdata --name dbdata docker-centos6.10-hadoop-spark

(2) Use --volumes-from in other containers to mount the data volume in the dbdata container

docker run -d --volumes-from dbdata --name db1 docker-centos6.10-hadoop-spark
docker run -d --volumes-from dbdata --name db2 docker-centos6.10-hadoop-spark

This enables data sharing between containers.

Enter these containers respectively and view the /volume1 and /volume2 directories.

4. Make your own image and publish it

Save container modifications and submit a new container image

docker commit centos1 centos111

Submit the existing container to form a new container image. Use docker images to see the centos111 image. This method creates a new container image.

View Mirror

docker images

REPOSITORY TAG IMAGE ID CREATED SIZE

centos111 latest d691a75ee371 23 minutes ago 501.5 MB

Create a container based on a new container image

docker run -d --name='centos111' centos111

View Container

docker inspect centos111

Exporting and importing images When you need to migrate an image from one machine to another, you need to export and import the image.
Machine A

docker save docker-centos6.10-hadoop-spark > docker-centos6.10-hadoop-spark2.tar

or

docker save -o docker-centos6.10-hadoop-spark docker-centos6.10-hadoop-spark2.tar

Use the scp command to copy docker-centos6.10-hadoop-spark2.tar to machine B in the same way as other methods.

docker load < docker-centos6.10-hadoop-spark2.tar

or

docker load -i docker-centos6.10-hadoop-spark2.tar

Publishing container images

docker push centos6.8-lamp1

Publish the container to the network.

5. Docker Network

When docker starts, it creates a virtual network interface named docker0 on the host machine. It will randomly select an unused host address and subnet mask from the private addresses defined in RFC 1918 and assign it to docker0. The default selection is 172.18.0.1/16. A 16-bit subnet mask provides 65534 IP addresses for the container.

Docker0 is not a normal network interface, but a virtual Ethernet bridge that automatically forwards packets between other network cards bound to it, allowing containers to communicate with the host and with each other.

Every time Docker creates a container, it creates a pair of peer interfaces, which are similar to the two ends of a pipe, where one side can receive data packets sent by the other side. Docker will connect one of the peer interfaces to the container as eth0 and will hold the other with a unique name like vethAQI2QT depending on the host's namespace. By binding all veth* interfaces to the docker0 bridge NIC, docker creates a shared virtual subnet between the host and all docker containers.

Docker NAT Network

By default, the docker container accesses the network through NAT. When docker starts, a virtual network interface named docker0 will be created on the host. Docker0 is just a virtual Ethernet bridge that automatically forwards data packets between other network cards bound to it. It enables containers and hosts to communicate with each other and between containers.

The gateway address of docker0 is 172.18.0.1, the mask is 16 bits, and 65534 IP addresses are provided.

In NAT mode, the virtual machine container can access the external network (outside the host machine), but machines other than the host machine cannot access the container intranet.

Docker Bridage Network

Docker containers can access the network through bridge.

In bridge mode, the virtual machine container can access the external network (outside the host machine), and machines outside the host machine can also access the container intranet.

6. Docker Pipework

The network function of Docker itself is relatively simple and cannot meet many complex application scenarios. Therefore, there are many open source projects used to improve the network functions of Docker, such as pipework, weave, flannel, etc.

Pipework is a docker network configuration tool developed by docker engineer Jérôme Petazzoni. It is implemented in more than 200 lines of shell and is easy to use.

Installing pipework

git clone https://github.com/jpetazzo/pipework
cp pipework/pipework /bin/

or

wget [http://172.17.1.240/docker/software/pipework](http://172.17.1.240/docker/software/pipework)
chmod a+x pipework
cp pipework /bin/

Running the container

docker run -d --net='none' --name='centos9' docker-centos6.10-hadoop-spark

Configure the container network and connect it to the bridge docker0; the gateway is specified by adding @ after the IP address.

pipework docker0 centos9 172.18.0.100/[email protected]

7. Docker network port mapping

If the container uses the docker0 virtual network, the container's network is 172.17.0.0/16. The container can access the external network through NAT; but the external network cannot access the internal network. If the container uses the br0 virtual network, the container and the server can be in the same network address segment; the container can access the external network; and the external network can also access the container network.
For containers using the docker0 virtual network, port mapping can be used to allow the external network to access certain ports of the container.

Running the container

docker run -d -p 38022:22 --name='centos10' docker-centos6.10-hadoop-spark

Connecting Containers

ssh localhost -p 38022

On other servers, you can access the container by accessing the physical server plus the port. You can map multiple ports at a time.
Running the container

docker run -d -p 38022:22 -p 38080:80 --name='centos11' docker-centos6.10-hadoop-spark

The implementation principle is to forward through iptables on the server. Of course, you can also forward the entire container IP address through iptables.

4. Conclusion

Because containers are process-level, they have many advantages over virtual machines.

(1) Starting an application in a fast container directly starts a process in the underlying system, rather than a process inside the virtual machine. Therefore, starting a container is equivalent to starting a process on the local machine instead of starting an operating system, which is much faster.

(2) Less resource usage: Containers only occupy the required resources and do not occupy unused resources; virtual machines are complete operating systems and therefore inevitably occupy all resources. In addition, multiple containers can share resources, while virtual machines have exclusive resources.

(3) Small size: A container only needs to contain the components it uses, while a virtual machine is a package of the entire operating system, so the container file is much smaller than the virtual machine file.
In short, containers are a bit like lightweight virtual machines that can provide a virtualized environment, but at a much lower cost.

The above is the full content of this article. I hope it will be helpful for everyone’s study. I also hope that everyone will support 123WORDPRESS.COM.