Linux disk space release problem summary

The /partition utilization of a server in IDC is full! Reached 100%! After checking, I found that one file was too large (80G), so after confirming with the relevant colleagues, I decisively deleted the file using rm -f. However, I found that after deleting the file, the disk space of the / partition was not released at all, and the usage rate was still 100%! Why is this? ?

[root@linux-node1 ~]# df -hFilesystem Size Used Avail Use% Mounted on/dev/mapper/VolGroup00-LogVol00 58G 7.8G 47G 100% /tmpfs 1.9G 0 1.9G 0% /dev/shm/dev/vda1 190M 72M 108M 40% /boot

Cause Analysis

In Linux system, deleting a file through rm or file manager will only unlink it from the directory structure of the file system, that is, only the link between the file and the system directory structure is deleted; if the file is open when it is deleted (a process is using the file, the file is locked by the process, or a process is writing data to the file, etc.), the process will still be able to read the file, that is, the file is not deleted. The reading status, so the disk space will always be occupied.

A file is stored in the file system in two parts: the data part and the pointer part. The pointer is located in the meta-data of the file system. After the data is deleted, the pointer is cleared from the meta-data, and the data part is stored on the disk. After the pointer corresponding to the data is cleared from the meta-data, the space occupied by the file data part can be overwritten and new content can be written. The reason why the space is not released after the file is deleted is that there is a process that is still writing content to the file. As a result, although the file is deleted, the pointer part corresponding to the file is not cleared from the meta-data due to the process locking. Since the pointer is not deleted, the system kernel believes that the file has not been deleted. Therefore, it is not surprising that the space has not been released when querying through the df command.

There are several solutions:

1. Use the lsof|grep deleted command to obtain a list of files that have been deleted but are still occupied by applications, and then kill the process that is still occupying the deleted files. It should be noted that if there are many processes using the deleted file, then using the first method to kill the process is a bit troublesome and risky. Because the kill process truncates the files in the proc file system, it can force the system to reclaim the files allocated to it that are in use. You must make sure that it will not affect the running process before using it. The application does not support this method well. When a file in use is truncated, unpredictable problems may occur.

2. Either stop or restart the application that uses the deleted file to allow the OS to automatically reclaim the disk space.

3. You can also restart the operating system, but this is not the best method. 4. For this kind of process that keeps writing logs to files, the best way to release the disk space occupied by the file is to clear the file online. In this way, not only can the disk space be released immediately, but the process can also continue to write logs to the file.

How to clear files online (such as /home/wangshibo.log):

# echo " " > /home/wangshibo.logb）# cat /dev/null > /home/wangshibo.logc）# > /home/wangshibo.log

There is another phenomenon of disk space usage problem: the disk space usage rate is not high when checking with the df -h command, and there is still a lot of free space. However, when creating files or writing data, an error message is displayed saying that the disk is full: " no space left on device "!

Generally, this problem is caused by the fact that the resource space in the partition directory is not actually released after being deleted. The specific processing flow is as follows:

1. First, run df -lh to check the disk usage. You will find that the Used space under the /data partition is very large, but it does not actually occupy that much space! 2. Find the partition where the deleted files are located, such as the /data partition. 3. Check all deleted files: lsof -n /data |grep deleted 4. Kill the delete process of these files to release space: lsof -n /data |grep deleted|awk '{print $2}'|xargs kill -95. Then run lsof -n /data |grep delete again, and there should be no results. 6. Note: When you just kill the deleted process, use df -h to check the /data partition. The Used space may be too large for a moment, but as the deleted process is killed, resources are gradually released, the Used space under the /data partition will gradually decrease, and the Avail space will gradually increase)

Most file systems reserve some space for emergency use (for example, when the hard disk is full). This ensures that some critical applications (such as databases) have some room when the hard disk is full, so that they will not crash immediately, giving the monitoring system and administrator some time to notice. However, sometimes this reserved hard disk space is a bit of a waste if it is not used.

In Linux systems, 5% of the disk space is usually reserved by default on ext2, ext3, and ext4 file systems. For example, if the disk is 2TB, this means that 100GB of space will be reserved. Doesn't this seem a bit wasteful? The default setting of 5% can be changed by using the "tune2fs" command, for example to reserve only 2% of the space. However, it is not recommended to set it to 0%, as it is unsafe in real-world environments.

# df -TFilesystem Type 1K-blocks Used Available Use% Mounted on/dev/vda1 ext4 41151808 4962148 34076228 13% /devtmpfs devtmpfs 1931468 0 1931468 0% /devtmpfs tmpfs 1941204 0 1941204 0% /dev/shmtmpfs tmpfs 1941204 652 1940552 1% /runtmpfs tmpfs 1941204 0 1941204 0% /sys/fs/cgrouptmpfs tmpfs 388244 0 388244 0% /run/user/0[root@ss-server ~]# df -hFilesystem Size Used Avail Use% Mounted on/dev/vda1 40G 4.8G 33G 13% /devtmpfs 1.9G 0 1.9G 0% /devtmpfs 1.9G 0 1.9G 0% /dev/shmtmpfs 1.9G 620K 1.9G 1% /runtmpfs 1.9G 0 1.9G 0% /sys/fs/cgrouptmpfs 380M 0 380M 0% /run/user/0

For example, the "/" partition above is the ext4 file system, and the default system reserves 5%, or 2G of space. Now you can use the "tune2fs" command to change the system reserved space to 2%.

# tune2fs -m 2 /dev/vda1tune2fs 1.42.9 (28-Dec-2013)Setting reserved blocks percentage to 2% (209704 blocks)

After execution, it is found that 1G of space is freed up in the "/" partition, and the system reserved space is now 2%.

/dev/shmtmpfs 1.9G 620K 1.9G 1% /runtmpfs 1.9G 0 1.9G 0% /sys/fs/cgrouptmpfs 380M 0 380M 0% /run/user/0 [root@ss-server ~]# df -hFilesystem Size Used Avail Use% Mounted on/dev/vda1 40G 4.8G 34G 13% /devtmpfs 1.9G 0 1.9G 0% /devtmpfs 1.9G 0 1.9G 0% /dev/shmtmpfs 1.9G 620K 1.9G 1% /runtmpfs 1.9G 0 1.9G 0% /sys/fs/cgrouptmpfs 380M 0 380M 0% /run/user/0

This is the end of this article about Linux disk space release. For more information about Linux disk space release, please search 123WORDPRESS.COM’s previous articles or continue to browse the following related articles. I hope you will support 123WORDPRESS.COM in the future!