Summary of data interaction between Docker container and host

Preface

When using Docker in a production environment, data often needs to be persisted or shared between multiple containers, which inevitably involves container data management operations.

Method 1: Docker cp command

docker cp: used to copy data between the container and the host.
Syntax # Copy files in container to host docker cp [OPTIONS] CONTAINER:SRC_PATH DEST_PATH|-
# Copy the host file to the container docker cp [OPTIONS] SRC_PATH|- CONTAINER:DEST_PATH

OPTIONS description:
-L : Keep links between source and target

Operation example:

Host to container

Container to host

Note: Although this method can also be used for management, the data is not integrated and is basically not used. It is only used for understanding.

Method 2: Docker data volume

1. What is volume

To understand Docker Volume, we first need to know how Docker's file system works. A Docker image is composed of multiple file systems (read-only layers). When we start a container, Docker loads the image layer and adds a read-write layer on top of it. If a running container modifies an existing file, the file will be copied from the read-only layer under the read-write layer to the read-write layer. The read-only version of the file still exists, but it has been hidden by the copy of the file in the read-write layer. When you delete the Docker container and restart it with the image, the previous changes will be lost. In Docker, the combination of a read-only layer and a read-write layer on top is called a Union FIle System.

In order to save (persist) data and share data between containers, Docker proposed the concept of Volume. Simply put, a Volume is a directory or file that bypasses the default union file system and exists on the host machine as a normal file or directory.

2. Characteristics of Data Volumes

• Data volumes can be shared and reused between containers

• Changes to the data volume will take effect immediately

• Updates to the data volume will not affect the image

• By default, data volumes persist even if the container is deleted

3. Data volume related operations

Usage: docker volume COMMAND
Manage volumes
Commands:
create Create a volume
inspect Display detailed information on one or more volumes
ls List volumes
prune Remove all unused local volumes
rm Remove one or more volumes

Creating a Data Volume

Usage: docker volume create [OPTIONS] [VOLUME]
Create a volume
Options:
 -d, --driver string Specify volume driver name (default "local")
 --label list Set metadata for a volume
 -o, --opt map Set driver specific options (default map[])

View all data volumes

Usage: docker volume ls [OPTIONS]
List volumes
Aliases:
 ls, list
Options:
 -f, --filter filter Provide filter values ​​(eg 'dangling=true')
 --format string Pretty-print volumes using a Go template
 -q, --quiet Only display volume names

View details of one or more volumes

Usage: docker volume inspect [OPTIONS] VOLUME [VOLUME...]
Display detailed information on one or more volumes
Options:
 -f, --format string Format the output using the given Go template

Deleting a single data volume

Usage: docker volume rm [OPTIONS] VOLUME [VOLUME...]
Remove one or more volumes. You cannot remove a volume that is in use by a container.
Aliases:
 rm, remove
Options:
 -f, --force Force the removal of one or more volumes

Delete all idle data volumes

Usage: docker volume prune [OPTIONS]
Remove all unused local volumes
Options:
 --filter filter Provide filter values ​​(eg 'label=<label>')
 -f, --force Do not prompt for confirmation

Operation example:

4. Data volume usage

The use of data volumes is similar to mounting a directory or file in Linux.

Users can create containers with data volumes by using the --volume/-v or --mount options of docker run, but the two parameters cannot be used at the same time.

In general, --mount is more explicit and verbose. The biggest difference is that the -v syntax combines all options in one field, while the --mount syntax separates them. For beginners, it is recommended to use --mount because it is easier to understand.

Usage: docker run [OPTIONS] IMAGE [COMMAND] [ARG...]
Run a command in a new container
Options:
 --volume list Bind mount a volume
 --volume-driver string Optional volume driver for the container
 --volumes-from list Mount volumes from the specified container(s)
 --mount mount Attach a filesystem mount to the container

4.1 --volume usage details

The parameter --volume (or -v for short) can only create a bind mount.

Command format:
-v [[HOST-OPTIONS:]CONTAINER-DIR[:OPTIONS]]]
HOST-OPTIONS:
VOLUME_NAME (data volume name)
ABSOLUTE-HOST-DIR (absolute path of the host file directory)
ABSOLUTE-HOST-FILE (absolute path of the host file) (If it is empty, the anonymous data volume will be mounted)
Options:
rw read-write (default)
ro Read only

4.1.1 Mounting by Data Volume Name

docker run -itd -v VOLUME_NAME:CONTAINER-DIR IMAGE [COMMAND] [ARG...]

Example:

Mount the data volume my_vol to the container's /data/docker/volume/my_vol directory

docker run -itd --name=vol1_ubuntu -v my_vol:/data/docker/volume/my_vol ubuntu /bin/bash

Note: If the data volume my_vol exists, it will be mounted directly. If it does not exist, Docker will automatically create the data volume and then mount it.

4.1.2 Mount according to the absolute path of the host file directory

docker run -it -v ABSOLUTE-HOST-DIR:CONTAINER-DIR IMAGE [COMMAND] [ARG...]

Example:

Mount the host file directory /opt/common/docker/volumes/my_vol2 to the container's /data/docker/volume/my_vol directory

docker run -itd --name=vol2_ubuntu -v /opt/common/docker/volumes/my_vol2:/data/docker/volume/my_vol ubuntu /bin/bash

Note: The file directory of the host machine must be an absolute path.

4.1.3 Mount according to the absolute path of the host file

docker run -itd -v VOLUME_NAME:CONTAINER-DIR IMAGE [COMMAND] [ARG...]

Example:

Mount the host file ~/.bash_history to the container's /.bash_history

docker run -itd --name=vol3_ubuntu -v ~/.bash_history:/.bash_history ubuntu /bin/bash

4.1.4 Mounting anonymous volumes

If the -v parameter does not add any host-related volume information, Docker will create an anonymous volume for mounting.

docker run -itd -v CONTAINER-DIR IMAGE [COMMAND] [ARG...]

Example:

Mount the anonymous volume to the container's /data/docker/volume/my_vol directory

docker run -itd --name=vol4_ubuntu -v /data/docker/volume/my_vol ubuntu /bin/bash

4.1.5 -v comprehensive operation example is as follows:

docker run -itd --name=vol_ubuntu \
-v my_vol:/data/docker/volume/my_vol1:ro \
-v /opt/common/docker/volumes/my_vol2:/data/docker/volume/my_vol2:rw \
-v /opt/common/docker/volumes/my_vol3.txt:/data/docker/volume/my_vol3.txt \
-v /data/docker/volume/my_vol4 \
ubuntu /bin/bash 

4.2 --mount usage details

--mount: consists of multiple <key>=<value> key-value pairs separated by commas. The order of the keys is arbitrary.

Command format:
--mount type=MOUNT-TYPE,<key>=<value>

For the --mount option, Docker currently provides three different types of data volumes to mount from the host to the container: volume, bind, and tmpfs.

The schematic diagrams of the three methods are shown below:

4.2.1 named mount

Normal data volumes (this type is the default), which are part of the host file system managed by Docker and are located in the /var/lib/docker/volumes directory by default;

--mount type=volume,source=<VOLUME-NAME>,destination=<CONTAINER-PATH>,volume-driver=<DRIVER-NAME>,volume-opt=<OPTION>=<VALUE>,readonly

<key>=<value> Description:
source: data volume source path, the keyword can be source or src, if this parameter is not present, Docker will automatically create an anonymous volume mount
VOLUME-NAME: data volume name
destination: the destination path of the data volume. The keyword can be destination, dst or target.
CONTAINER-PATH: The mount path of the data volume in the container
volume-driver: specifies the data volume driver, the default value is "local"
DRIVER-NAME: driver name
volume-opt: optional, data volume driver parameter options
OPTION: key value
VALUE: value
readonly: optional, set the data volume to be mounted in the container with read-only permissions, the default is readable and writable

Code example:

docker service create --mount 'type=volume,src=<VOLUME-NAME>,dst=<CONTAINER-PATH>,volume-driver=local,volume-opt=type=nfs,volume-opt=device=<nfs-server>:<nfs-path>,"volume-opt=o=addr=<nfs-address>,vers=4,soft,timeo=180,bg,tcp,rw"' --name myservice <IMAGE>

4.2.2bindmount

Bind data volumes, files or directories to mount, which means they can be stored anywhere on the host system;

--mount type=bind,source=<HOST-PATH>,destination=<CONTAINER-PATH>,bind-propagation=<PG-TYPE>,readonly

<key>=<value> Description:
source: data volume source path, the keyword can be source or src, and before execution, make sure the file or file directory exists, otherwise an error will occur.
HOST-PATH: The absolute path of the host file or directory
destination: the destination path of the data volume. The keyword can be destination, dst or target.
CONTAINER-PATH: The mount path of the data volume in the container
bind-propagation: optional,
PG-TYPE: Optional values: rprivate, private, rshared, shared, rslave, slave.
readonly: optional, set the data volume to be mounted in the container with read-only permissions. The default is readable and writable, so this parameter is not added.

*Note: This usage is not supported in Dockerfile because Dockerfile is intended for porting and sharing. However, the path formats of different operating systems are different, so it is not currently supported.

Code example:

docker run -d -it --name devtest --mount type=bind,source="$(pwd)"/target,target=/app --mount type=bind,source="$(pwd)"/target,target=/app2,readonly,bind-propagation=rslave nginx:latest

4.2.2 tmpfs mount

Temporary data volumes are mounted and stored in the host system's memory, but are not written to the host's file system;

--mount type=tmpfs,destination=<CONTAINER-PATH>,tmpfs-size=<SIZE-VALUE>,tmpfs-mode=<MODE-VALUE>,readonly

<key>=<value> Description:
destination: the destination path of the data volume. The keyword can be destination, dst or target.
CONTAINER-PATH: The mount path of the data volume in the container
tmpfs-size: Optional, the size of the tmpfs mount (in bytes), the default in Linux is infinite.
SIZE-VALUE: value
tmpfs-mode: Optional, octal file mode of tmpfs, the default value in Linux is "1777".
MODE-VALUE: Value

Note: This feature is only available when running Docker on Linux.

Code example:

docker run -d -it --name tmptest --mount type=tmpfs,destination=/app,tmpfs-size=1024,tmpfs-mode=1770 nginx:latest

4.3 Difference between -v and --mount

(1) --mount can support the creation of data volumes for cluster services, while -v cannot.

(2) Mount a file or directory. If it does not exist before mounting, -v docker will automatically create it, but --mount will not (an error will be reported).

4.4 Supplementary Notes

(1) (--mount basically includes the optional attributes of --volume) The official recommendation is to use the --mount method. The original text is as follows:

Even though there is no plan to deprecate --volume , usage of --mount is recommended.

(2) Official document link:

https://docs.docker.com/storage/volumes/

https://docs.docker.com/engine/reference/commandline/service_create/

Method 3: Docker data volume container

A data volume container is also a container, but its purpose is to provide data volumes for other containers to mount. If users need to share some continuously updated data between multiple containers, the easiest way is to use a data volume container.

1. Basic use of data volume containers 1.1 Creating a data volume container

Create a data volume container dbdata, and create a data volume in it and mount it to the /dbdata directory:

docker run -it -v /dbdata --name db_data ubuntu

1.2 Mounting other containers

Use --volumes-from in other containers to mount the data volumes in the dbdata container

docker run -it --volumes-from db_data --name db1 ubuntu
docker run -it --volumes-from db_data --name db2 ubuntu

db1 and db2 share data through db_data

2. Use data volume containers to back up, restore, and migrate data volumes

Data volumes can be used to back up, restore, and migrate data.

2.1 Backup

First, use the --volumes-from flag to create a container that mounts the dbdata container volume and mounts the current /backup directory from the local host into the container. The command is as follows:

sudo docker run --volumes-from dbdata -v $(pwd):/backup ubuntu tar cvf /backup/backup.tar /dbdata

After the container is started, the tar command is used to back up the dbdata volume to the local /backup/backup.tar.

2.2 Recovery

If you want to restore data to a container, first create a container dbdata2 with a data volume.

sudo docker run -v /dbdata --name dbdata2 ubuntu /bin/bash

Then create another container, mount the dbdata2 container, and use untar to decompress the backup file into the mounted container volume.

sudo docker run --volumes-from dbdata2 -v $(pwd):/backup busybox tar xvf /backup/backup.tar

Disclaimer: The content of this blog post is purely personal understanding. If you have any objections, please comment.

This concludes this article on the data interaction between Docker containers and the host. For more information on data interaction between Docker containers and the host, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope you will support 123WORDPRESS.COM in the future!