Detailed explanation of Docker daemon security configuration items

This article will introduce you to the relevant security configuration items of the Docker daemon process.

1. Test environment

1.1 Install CentOS 7

CentOS Linux release 7.7.1908 (Core)

Upgrade the kernel and reboot

# yum update kernel
[root@localhost docker]# uname -a
Linux localhost 3.10.0-1062.12.1.el7.x86_64 #1 SMP Tue Feb 4 23:02:59 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
[root@localhost docker]# cat /etc/redhat-release
CentOS Linux release 7.7.1908 (Core)

1.2 Install Docker CE 19.03

# yum install -y yum-utils device-mapper-persistent-data lvm2
# yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

# yum install -y docker-ce

[root@localhost docker]# docker --version
Docker version 19.03.8, build afacb8b

2. Daemon security configuration

There is no configuration file by default. You need to create /etc/docker/daemon.json separately. The following configurations are all configured in this file, and are local test examples.

{
 "icc": false,
 "log-level": "info",
 "log-driver": "json-file",
 "log-opts": {
 "max-size": "10m",
 "max-file":"5",
 "labels": "somelabel",
 "env": "os,customer"
 },
 "iptables": true,
 "userns-remap": "default",
 "userland-proxy": false,
 "experimental": false,
 "selinux-enabled": true,
 "live-restore": true,
 "no-new-privileges": true,
 "cgroup-parent": "/foobar",
 "seccomp-profile": "/etc/docker/seccomp/default-no-chmod.json",
 "tls": true,
 "tlsverify": true,
 "tlscacert": "/etc/docker/CA/ca.pem",
 "tlscert": "/etc/docker/CA/server-cert.pem",
 "tlskey": "/etc/docker/CA/server-key.pem"
}

2.1 Configure access to the Docker daemon via HTTPS and certificate authentication

Server Certificates

Create a HOST and define the domain (IP is also OK). The corresponding certificate will be generated according to the domain, which is generally used to register the CN in the certificate:

Create a certificate directory:

$ mkdir -p /etc/docker/dockerd/CA && cd /etc/docker/dockerd/CA

Generate a key certificate and fill in the key certificate password twice:

$ openssl genrsa -aes256 -out ca-key.pem 4096

To generate a CA certificate, you need to enter the basic information of the registration certificate:

$ openssl req -new -x509 -days 365 -key ca-key.pem -sha256 -out ca.pem

Create a server certificate:

$ openssl genrsa -out server-key.pem 4096

$ openssl req -subj "/CN=localhsot" -sha256 -new -key server-key.pem -out server.csr

Set the IP address specified by the certificate:

$ echo subjectAltName = DNS:localhost,IP:127.0.0.1 >> extfile.cnf

Set the extended usage property of the Docker daemon key to be used for server authentication only:

$ echo extendedKeyUsage = serverAuth >> extfile.cnf

Generate server cert certificate:

$ openssl x509 -req -days 3650 -sha256 -in server.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out server-cert.pem -extfile extfile.cnf

Client Certificates

Create a client certificate: (still in the current directory)

$ openssl genrsa -out key.pem 4096
$ openssl req -subj '/CN=localhost' -new -key key.pem -out client.csr

To make the key suitable for client authentication, create an extended configuration file:

$ echo extendedKeyUsage = clientAuth >> extfile.cnf

Generate the client cert certificate:

$ openssl x509 -req -days 3650 -sha256 -in client.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out cert.pem -extfile extfile.cnf

use

Grant corresponding permissions to the certificate:

$ chmod -v 0400 ca-key.pem key.pem server-key.pem
$ chmod -v 0444 ca.pem server-cert.pem cert.pem

[root@localhost CA]# ls
ca-key.pem ca.pem ca.srl cert.pem client.csr extfile.cnf key.pem server-cert.pem server.csr server-key.pem

Server configuration /etc/docker/daemon.json

"tls": true,
"tlsverify": true,
"tlscacert": "/etc/docker/CA/ca.pem",
"tlscert": "/etc/docker/CA/server-cert.pem",
"tlskey": "/etc/docker/CA/server-key.pem"

Client Configuration

Set the client certificate to the server and place it in the appropriate location:

$ cp -v {ca,cert,key}.pem ~/.docker
$ export DOCKER_HOST=tcp://$HOST:2376 DOCKER_TLS_VERIFY=1

The simulation test is performed as follows:

$ curl https://$HOST:2376/images/json \
 --cert ~/.docker/cert.pem \
 --key ~/.docker/key.pem \
 --cacert ~/.docker/ca.pem
 
[{"Containers":-1,"Created":1540777343,"Id":"sha256:55e7b305dc477345434ce3bd3941940481f982eea31c8f28c0670d59c63d544b","Labels":nu

2.2 Using namespace isolation technology

Namespace is an isolation technology. Docker uses isolation technology to open a specific namespace and create some special processes. However, the use of namespace is conditional. The system will create dockremap and map it to the container through /etc/subuid and the id value corresponding to /etc/subuid; in reality, the normal permission of dockremap is still used to achieve the effect of automatic isolation.

First modify /etc/sysctl.conf

# echo "user.max_user_namespaces=15076" >> /etc/sysctl.conf

Add the configuration item "userns-remap": "default" in /etc/docker/daemon.json

You need to be careful when modifying this configuration. If you have already deployed a Docker environment, after enabling this option, you will switch to an isolated environment and the previous Docker container will not be available!

[root@localhost docker]# cat /etc/subuid
dockremap:100000:65536

2.3 Setting up Docker partitions

Create a separate partition for the container. The default partition is in \var\lib\docker\ , which contains local images, containers, networks, and other related things.

[root@localhost docker]# ls /var/lib/docker

100000.100000 builder buildkit containers image network overlay2 plugins runtimes swarm tmp trust volumes
The default partition location can be configured using "data-root": "".

2.4 Limiting traffic between default bridge containers

When the Docker service is started, a forwarding policy is added to the FORWARD chain of iptables by default. The policy is either ACCEPT or DROP, depending on whether --icc=true (default) or --icc=false is configured. If you manually specify --iptables=false then no iptables rules will be added.

By default, all network communication is allowed between containers on the same host on the default bridge. If this is not needed, restrict communication between all containers. Link together specific containers that need to communicate, or create a custom network and join only the containers that need to communicate to that custom network.

Configure to limit traffic between containers on the default bridge "icc": false

2.5 Configuring Logging

Configure centralized remote logging, set the log process --log-level level to info, log format to json, local logging

"log-level": "info",
"log-driver": "json-file",
"log-opts": {
 "max-size": "10m",
 "max-file":"5",
 "labels": "somelabel",
 "env": "os,customer"
},

Configuring Remote Logging

The Docker logging driver receives container logs and forwards them to a remote destination or file. The default logging driver is json-file . It stores container logs in JSON format on local disk. Docker has a plugin architecture for logging, so there are plugins for both open source and commercial tools:

Journald – Stores container logs in the system journal.
Syslog Driver – Supports UDP, TCP, TLS
Fluentd – Supports connecting TCP or Unix sockets to fluentd
Splunk – HTTP/HTTPS forwarding to Splunk server
Gelf – UDP log forwarding to Graylog2

Example fluent

{
 "log-driver": "fluentd",
 "log-opts": {
 "fluentd-address": "fluentdhost:24224"
 }
 }

Using syslog

{
 "log-driver": "syslog",
 "log-opts": {
 "syslog-address": "udp://1.2.3.4:1111"
 }
}

2.6 Setting ulimit

{
 "default-ulimits": {
 "nofile": {
  "Name": "nofile",
  "Hard": 64000,
  "Soft": 64000
 }
 }
}

2.7 Setting cgroup

The --cgroup-parent option allows setting the default cgroup parent to use for containers. If this option is not set, it defaults to /docker for the fs cgroup driver and system.slice for the systemd cgroup driver.

If the cgroup has a forward slash ( / ), the cgroup is created under the root cgroup, otherwise the cgroup is created under the daemon cgroup.

Assuming the daemon runs in cgroup daemoncgroup, then --cgroup-parent=/foobar creates a cgroup in /sys/fs/cgroup/memory/foobar, while with --cgroup-parent=foobar it creates a cgroup in /sys/fs/cgroup/memory/daemoncgroup/foobar .

The systemd cgroup driver has different rules for --cgroup-parent. Systemd represents the hierarchy in slices, where the name of the slice encodes the position in the tree. Therefore, --cgroup-parent of systemd cgroup should be the slice name. The name can consist of a series of names separated by dashes that describe the path from the root slice to the slice. For example, --cgroup-parent=user-ab.slice means that the container's memory cgroup is created in /sys/fs/cgroup/memory/user.slice/user-a.slice/user-ab.slice/docker-<id>.scope <id>.scope.

It can also be set using the container runtime, using the --cgroup-parent option on docker create and docker run . This takes precedence over the --cgroup-parent option on the daemon.

2.8 Configuring seccomp

The test configuration file used prohibits the use of the chmod command in Docker

https://github.com/docker/labs/blob/master/security/seccomp/seccomp-profiles/default-no-chmod.json
[root@localhost docker]# docker run --rm -it alpine sh
/ # ls bin etc lib mnt proc run srv tmp var
dev home media opt root sbin sys usr / # touch foo.sh
/ # chmod +x foo.sh
chmod: foo.sh: Operation not permitted
/ # exit

It can actually prohibit, allow, and warn certain system-related calls, refer to: https://github.com/torvalds/linux/blob/master/arch/x86/entry/syscalls/syscall_64.tbl

2.9 Configuring containers with no daemon support

--live-restore ensures that the docker daemon is shut down without affecting the container.

During the test, after shutting down the docker daemon, the nginx container can still provide normal access.

2.10 Disable experimental features of Docker

Set "experimental": false

2.11 Limiting container privilege escalation via suid or sgid

The no-new-privileges security option prevents application processes within a container from acquiring new privileges during execution.

For example, if there is a program with the setuid/setgid bit set in the image, such as sudo, the process in the container also has (file) permissions to execute that program. Any attempt to gain privileges through facilities such as setuid/setgid will be denied.

3. Daemon configuration example (Linux)

{
 "authorization-plugins": [], //Access authorization plug-in "data-root": "", //The root directory for persistent storage of docker data, the default is /var/lib/docker
 "dns": [], //DNS server"dns-opts": [], //DNS configuration options, such as port, etc. "dns-search": [], //DNS search domain name"exec-opts": [], //Execution options"exec-root": "", //The root directory of the executed file"experimental": false, //Whether to enable experimental features"features": {}, //Enable or disable specific features. For example: {"buildkit": true} makes buildkit the default Docker image builder.
 "storage-driver": "",//Storage driver type "storage-opts": [],//Storage options "labels": [],//Key-value pair to mark docker metadata "live-restore": true,//Whether to keep the container alive when dockerd hangs up (avoiding container exit caused by docker service exception)
 "log-driver": "json-file", //Container log driver "log-opts": {
 "max-size": "10m",
 "max-file":"5",
 "labels": "somelabel",
 "env": "os,customer"
 }, //Container log options "mtu": 0, //Set container network MTU (maximum transmission unit)
 "pidfile": "", //daemon PID file location "cluster-store": "", //cluster storage system URL
 "cluster-store-opts": {},//Configure cluster storage"cluster-advertise": "",//External address name"max-concurrent-downloads": 3,//Set the maximum concurrency of each pull process"max-concurrent-uploads": 5,//Set the maximum concurrency of each push process"default-shm-size": "64M",//Set the default shared memory size"shutdown-timeout": 15,//Set the shutdown timeout limit"debug": true,//Turn on debug mode"hosts": [],//Listening address of the dockerd daemon process"log-level": "",//Log level"tls": true,//Turn on the transport layer security protocol TLS
 "tlsverify": true, // Enable transport layer security protocol and verify remote address "tlscacert": "", // CA signature file path "tlscert": "", // TLS certificate file path "tlskey": "", // TLS key file path "swarm-default-advertise-addr": "", // swarm external address "api-cors-header": "", // Set CORS (Cross-origin resource sharing) header "selinux-enabled": false, // Enable selinux (mandatory access control for users, processes, applications, and files)
 "userns-remap": "", //Set users/groups for user namespace "group": "", //Docker group "cgroup-parent": "", //Set the parent class of cgroup for all containers "default-ulimits": {
 "nofile": {
  "Name": "nofile",
  "Hard": 64000,
  "Soft": 64000
 }
 }, //Set ulimit for all containers
 "init": false, //Container performs initialization to forward signals or control (reap) processes "init-path": "/usr/libexec/docker-init", //Path to the docker-init file "ipv6": false, //Support IPV6 network "iptables": false, //Open firewall rules "ip-forward": false, //Open net.ipv4.ip_forward
 "ip-masq": false, // Enable ip masking (a technique for rewriting source or destination IP addresses when IP packets pass through a router or firewall)
 "userland-proxy": false, //Userland proxy "userland-proxy-path": "/usr/libexec/docker-proxy", //Userland proxy path "ip": "0.0.0.0", //Default IP
 "bridge": "", //Attach the container to the bridge network "bip": "", //Specify the bridge IP
 "fixed-cidr": "",//(ipv4) subnet division, that is, limiting the range of ip address allocation, to control the network segment to which the container belongs to achieve network access between containers (same host or different hosts)"fixed-cidr-v6": "",//(ipv6) subnet division"default-gateway": "",//Default gateway"default-gateway-v6": "",//Default ipv6 gateway"icc": false,//Communication between containers"raw-logs": false,//Raw logs (no color, full timestamp)
 "allow-nondistributable-artifacts": [],//Registry repositories submitted for products that are not distributed externally"registry-mirrors": [],//Registry repository mirror acceleration address"seccomp-profile": "",//seccomp configuration file"insecure-registries": [],//Configure non-https registry address"no-new-privileges": false,//Prohibit new priorities"default-runtime": "runc",//OCI Alliance (The Open Container Initiative) default runtime environment"oom-score-adjust": -500,//Priority for killing due to memory overflow (-1000~1000)
 "node-generic-resources": ["NVIDIA-GPU=UUID1", "NVIDIA-GPU=UUID2"], //Resource nodes published to the public "runtimes": {
 "cc-runtime": {
  "path": "/usr/bin/cc-runtime"
 },
 "custom": {
  "path": "/usr/local/bin/my-runc-replacement",
  "runtimeArgs": [
  "--debug"
  ]
 }
 }, //Runtime "default-address-pools":[
 {"base":"172.80.0.0/16","size":24}, //Default DHCP assigned address {"base":"172.90.0.0/16","size":24}
 ]
}

Summarize

This is the end of this article about the detailed explanation of the Docker daemon security configuration project. For more relevant Docker daemon content, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope everyone will support 123WORDPRESS.COM in the future!