This article will help you thoroughly understand the specific use of cgroup in Docker

Preface

The process uses CPU, memory, disk and other computing resources or storage resources in the system relatively arbitrarily. We hope to limit the process resource utilization and track the use of process resources. This makes the emergence of cgroup possible, which is used to group processes uniformly and monitor and control resources on the basis of groups.

What is cgroup

Linux CGroup (Linux Control Group) is actually a function of the Linux kernel. It is a mechanism under Linux to manage processes by group. It was first initiated by Google engineers Paul Menage and Rohit Seth in 2006 and was originally named process container. After 2007, with the introduction of containers, it was renamed to cgroup to avoid confusion and was merged into kernel version 2.6.24.
From the user level's perspective, cgroup technology organizes all processes in the system into independent trees. Each tree contains all the processes of the system. Each node of the tree is a process group, and each tree is associated with one or more subsystems. The tree is mainly used to group processes, and the subsystem is used to operate on these groups.

Composition of cgroup

Cgroup mainly consists of the following two parts

• Subsystem: A subsystem is a kernel module. After it is associated with a cgroup tree, it will perform specific operations on the tree nodes. The subsystem is often called a "resource controller" because it is mainly used to schedule or limit the resources of each process group, but this statement is not entirely accurate, because sometimes we group processes just to do some monitoring and observe their status, such as the perf_event subsystem.
• Hierarchy: A hierarchy can be understood as a cgroup tree. Each node of the tree is a process group, and each tree is associated with multiple subsystems. In a tree, all processes in the Linux system are included, but each process can only belong to one node (process group). There can be many cgroup trees in the system, each tree is associated with a different subsystem, and a process can belong to multiple trees, that is, a process can belong to multiple process groups, which are associated with different subsystems.

You can check which subsystem associations the current system supports by viewing the /proc/cgroup directory

The first column: indicates the subsystem name

The second column: indicates the ID of the associated cgroup tree. If multiple subsystems are associated with the same cgroup tree, their fields will be the same. For example, cpuset, cpu and cpuacct in the figure.

The third column: indicates the number of process groups in the cgroup tree associated with the subsystem, that is, the number of nodes on the tree.

Functions provided by cgroup

It provides the following functions

• Resource limitation: Resource usage limitation
• Prioritization: Priority control
• Accounting: some auditing or statistics
• Control: Suspend the process and resume the execution process

Generally we can use cgroup to do the following things

• Isolate a set of processes (such as all MySQL processes) and limit the resources they use, such as bound core limits
• Allocate memory for this set of processes
• Allocate enough bandwidth and storage limits for this set of processes
• Restrict access to certain devices

Cgroup appears as a file system in Linux. Run the following command

After the mount is successful, you can see that there is a cgroup directory under /sys/fs, which contains many subsystems. For example, cpu, cpuset, blkio, etc.
Then create a subdirectory test in the /sys/fs/cgroup/cpu directory. At this time, you will find that there are many more files in this directory.

Limiting CPU in a cgroup

In cgroup, the CPU-related subsystems include cpusets, cpuacct, and cpu.
Among them, cpuset is mainly used to set the affinity of the CPU. It can limit the processes in the cgroup to run only on the specified CPU, or not to run on the specified CPU. At the same time, cpuset can also set the affinity of the memory. cpuacct contains statistics about the CPU used by the current cgroup. Here we only talk about the following CPUs.

Then we create a subgroup under /sys/fs/cgroup/cpu, the file list under this directory

cpu.cfs_period_us is used to configure the length of the time period, and cpu.cfs_quota_us is used to configure the amount of CPU time that the current cgroup can use within the set period length. The two files work together to set the upper limit of CPU usage. The units of both files are microseconds (us). The value range of cpu.cfs_period_us is 1 millisecond (ms) to 1 second (s). The value of cpu.cfs_quota_us can be greater than 1ms.
Let's take an example to explain how to use CPU limit. If we write an infinite loop

When running, use top to check that the occupancy rate has reached 100%

We execute the following command to set cfs_quota_us

echo 20000 > /sys/fs/cgroup/cpu/test/cpu.cfs_quota_us

This command means reducing the CPU utilization of the process by 20% and adding the process PID to the cgroup.

Execute top again and you can see that the CPU utilization has dropped.

Limiting memory in a cgroup

If the code has bugs, such as memory leaks, it will drain the system memory and cause other programs to behave abnormally due to insufficient memory allocation. If the system is configured with a swap partition, the system will use a large amount of the swap partition, causing the system to run very slowly.
The main control of cgroup on process memory is as follows:

• Limit the total memory used by all processes in the cgroup
• Limit the total amount of physical content + swap used by all processes in the cgroup
• Limit the total amount of kernel memory and other kernel resources that can be used by all processes in the cgroup (CONFIG_MEMCG_KMEM).

Limiting kernel memory here means limiting the kernel resources currently used by the cgroup, including the kernel space occupied by the current process, the memory space occupied by the socket, etc. When memory is tight, the current cgroup can be prevented from continuing to create processes and requesting more kernel resources from the kernel.

The following example shows how cgroup can control memory.

#include <iostream>
#include <sys/types.h>
#include <cstdlib>
#include <cstdio>
#include <string.h>
#include <unistd.h>

#define CHUNK_SIZE 512


int main()
{
   int size = 0;
   char *p = nullptr; 
   while(1)
   {
          if((p = (char*)malloc(CHUNK_SIZE))==nullptr)
          {
              break;
         }

      memset(p, 0, CHUNK_SIZE);
       printf("[%u]-- [%d]MB is allocated ", getpid(), ++size);
       sleep(1);
   }
    
   return 0;
}

First, create a subdirectory under /sys/fs/cgroup/memory to create a subcgroup. For example, here we create a test directory

$mkdir /sys/fs/cgroup/memory/test

The test directory contains the following files

The function of each file is briefly introduced below:

Then set the limit by writing the file memory.limit_in_bytes. Here, a limit of 5M is set, as shown in the figure below

Add the above example process to this cgroup, as shown in the following figure

To avoid being affected by the swap space, set swappiness to 0 to prohibit the current cgroup from using swap, as shown in the following figure

When the physical memory reaches the upper limit, the system's default behavior is to kill the process in the cgroup that continues to request memory. So how do you control this behavior? That is to configure memory.oom_control. This file contains a flag that controls whether OOM-killer is started for the current cgroup. If 0 is written to this file, OOM-killer will be started. When the kernel cannot allocate enough memory to the process, it will kill the process directly. If 1 is written to this file, OOM-killer will not be started. When the kernel cannot allocate enough memory to the process, it will pause the process until there is free memory and then continue to run. At the same time, memory.oom_control also contains a read-only under_oom field, which is used to indicate whether the current state has entered the OOM state, that is, whether there is a process that has been suspended. There is also a read-only killed_oom field that indicates whether any process has been killed.

Limit the number of cgoup processes

There is a subsystem called pids in cgroup, which limits the total number of tasks that can be created in the cgroup and all its descendant cgroups. The task here refers to the process created by the fork and clone functions. Since the clone function can also create threads, the task here also includes threads.
The cgroup tree has been mounted before, so we will directly create a child cgroup here and name it test. The command is shown below

Let's take a look at the files in the test directory

Where pids.current indicates the total number of processes in the current cgroup and all its grandchild cgroups.

pids.max The maximum number of processes allowed to be created by the current cgroup and all its grandchild cgroups.

Let's do an experiment and set pids.max to 1

Then add the current bash process to the cgroup

Run a command at random. Since pids.current is equal to pids.max in the current window, the process creation fails.

The pids.current and pids.max in the current cgroup represent all processes of the current cgroup and all descendant cgroups, so the pids.max size in the descendant cgroup cannot exceed the size in the parent cgroup. What happens if it exceeds? We set pids.max to 3

The current number of processes is 2

Reopen a shell window, create a grandchild cgroup, and set pids.max to 5

Write the current shell's bash process to croup.procs

Go back to the original shell window and execute a command at random to see that the execution failed

As you can see, the number of processes in a child cgroup is not only limited by its own pids.max, but also by the pids.max of the ancestor cgroup.

This is the end of this article about how to thoroughly understand the specific use of cgroup in Docker. For more relevant Docker cgroup content, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope everyone will support 123WORDPRESS.COM in the future!