Detailed explanation and summary of the use of Linux scheduled task Crontab command

The crontab command is used by Unix and Linux to set up periodic execution instructions. It is a very common technology on the Internet. Many tasks are set up in crontab for cyclic execution.

If crontab is not used, the task will be a resident program, which places high demands on your program. One requirement is that your program should be up and running 24/7, and the other is that your scheduling program should be reliable. In actual work, 90% of programs do not need to spend so much time and energy to solve the above two problems. You only need to write your own business logic and schedule it through Crontab, an industrial-grade program. There should be no doubt about the reliability and robustness of Crontab.

A simple introduction to crontab command

Suppose I want to set a task to synchronize data every minute. The path of this synchronization script is /home/blue/do/rsyncfile.sh. Then I can configure it like this. As user blue, enter in the terminal:

crontab -e
# You will then enter the vi editing screen to allow you to edit your work! Notice that each job is a row.
#Time-sharing day, month, week|<================The complete command line for the task * * * * * /home/blue/do/rsyncfile.sh

By default, any user who is not included in /etc/cron.deny can directly issue "crontab -e" to edit his own routine commands! The whole process is just as mentioned above. You will enter the vi editing screen and then edit one line at a time. After editing, enter ":wq" to save and exit vi!

If we need to modify the script to run data synchronization every 5 minutes, we can also use crontab -e to enter the editor:

*/5 * * * * /home/blue/do/rsyncfile.sh

If there is a problem with the server and the data is not synchronized for one day, we need to supplement the data. Assume that the script for supplementing the data is /home/blue/do /rsyncfile_day.sh. However, the daytime is the peak period and there are not many users at night, which is the off-peak period. Supplementing the data will occupy a lot of bandwidth, especially during the day, which will affect normal business. Therefore, we can generally let the data supplement task start at 2 am, and then use crontab -e to enter the editor:

0 2 1 4 * /home/blue/do/rsyncfile_day.sh

In this way, our data supplement script will be started at 2:00 am on April 1st.

Synchronizing data is a very common task in Internet companies. Here you can see the charm of crontab. You only need to write the simplest business logic and leave the scheduling to crond to complete a task with high reliability. If you want to write this kind of scheduling program yourself, I don’t know how much effort it will take to make it reliable and stable.

Syntax of crontab Command

crontab [-u username] [-l|-e|-r]
Options and parameters:
-u: Only root can perform this task, that is, to help other users create/remove crontab task schedules;
-e: edit crontab job content -l: view crontab job content -r: remove all crontab job content. If you only want to remove one item, please use -e to edit

Query the user's current crontab content:

crontab -l
*/5 * * * * /home/blue/do/rsyncfile.sh
0 2 1 4 * /home/blue/do/rsyncfile_day.sh

Clear the user's current crontab:

crontab -r
crontab -l
no crontab for blue

If you want to delete a crontab task of the current user, use crontab -e to enter the editor and then delete the corresponding task.

Limitations of crontab Command

/etc/cron.allow : Write the accounts that can use crontab into it. Users not in this file cannot use crontab;

/etc/cron.deny : Accounts that are not allowed to use crontab are written into it. If the user is not recorded in this file, he can use crontab.

In terms of priority, /etc/cron.allow takes precedence over /etc/cron.deny. Judging from the above, only one of these two files is selected to restrict. Therefore, it is recommended that you only keep one to avoid affecting your judgment on the configuration! Generally speaking, the system retains /etc/cron.deny by default. You can write the user you do not want to run crontab into /etc/cron.deny, one account per line!

Detailed explanation of crontab configuration file

crontab -e is designed for the user's cron. If it is a "routine task of the system", you need to edit the /etc/crontab file.

That is crontab -e. This crontab is actually the running file /usr/bin/crontab, but /etc/crontab is a "plain text file" and must be edited as root.

First, let's take a look at the contents of the crontab file

cat /etc/crontab
 
# /etc/crontab: system-wide crontab
# Unlike any other crontab you don't have to run the `crontab'
# command to install the new version when you edit this file
# and files in /etc/cron.d. These files also have username fields,
# that none of the other crontabs do.
 
SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
 
# mh dom monitor dow user command
17 * * * * root cd / && run-parts --report /etc/cron.hourly
25 6 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
47 6 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly )
52 6 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )

This file is almost exactly the same as the content of the crontab -e we just issued! There are just a few places that are different

PATH=....:

Here is the search path for entering the run file! Using the default path configuration is sufficient!

17 * * * * root cd / && run-parts --report /etc/cron.hourly:

This /etc/crontab is pre-configured with four tasks, which are performed once every hour, every day, every week and every month! However, what follows the five columns is not a command, but a new column, which is the "identity that runs the following commands"! This is different from the user's crontab -e . Since the user's own crontab does not need to specify an identity, the identity must be specified in /etc/crontab ! According to the above table, the system default routine work is performed as root.

So what is the following string of commands? You can use which run-parts to search for it, it is actually a bash script! If you go directly to /usr/bin/run-parts , you will find that this command will grab all the files in the "directory" that follows it and run them! This means "If you want the system to actively run a command for you every hour, write the command into a script and place the file in the /etc/cron.hourly/ directory"!

Now do you know how the system schedules a bunch of routine tasks by default? If you issue ll /etc/cron.daily , you will see a bunch of files. Those files are the scripts provided by the system, and these scripts will start running at 6:25 am every day!

Suppose you want to make a directory so that the system can run all executable files in this directory every 2 minutes. You can write the following line in /etc/crontab:

*/2 * * * * root run-parts /etc/cron.min

Of course, the /etc/cron.min directory must exist! What if I just need to run a "program" and don't need a directory? What should I do? For example, when detecting network traffic, we want to detect and analyze it every five minutes. We can write it like this:

*/5 * * * * root /bin/mrtg /etc/mrtg/mrtg.cfg

how! Creating routine commands is easy! If you are a system administrator and your job is routine system maintenance, you can directly modify the /etc/crontab file! It is convenient and easy to manage!

The principle of crontab command

When a user uses the crontab command to create a task schedule, the task will be recorded in /var/spool/cron/ and will be identified by the account number! For example, after blue uses crontab, his work will be recorded in /var/spool/cron/blue ! But please note, do not use vi to edit the file directly, because cron may not run due to input syntax errors! In addition, every job run by cron will be recorded in the /var/log/cron /var/log/cron login file!

The minimum detection limit of the crond service is "minutes", so "cron will read the data content in /etc/crontab and /var/spool/cron once a minute". Therefore, as long as you edit the /etc/crontab file and save it, the cron configuration will run automatically!

Note: crontab under Linux will automatically help us re-read the routine tasks in /etc/crontab every minute. However, for some reasons or in other Unix systems, crontab is read into the memory, so after you modify /etc/crontab, it may not run immediately. At this time, please restart the crond service! /etc/init.d/crond restart or service crond restart

Detailed explanation of crontab command format

The format of each task (each line) has six fields, the meanings of these six fields are:

The more interesting one is the "Zhou"! When the number of the week is 0 or 7, it means "Sunday"! In addition, there are some auxiliary characters, which are roughly as follows:

Zhou and the sun and moon cannot coexist

Another point to note is: "You can use weeks or days and months as the cycle, but you cannot work on a 'month and day of the week' model." This means that you cannot write a task schedule like this:

30 12 11 9 5 root echo "just test" <==This is wrong

You originally thought that this task would be performed only if September 11th falls on a Friday. Unfortunately, the system may determine that this task should be performed every Friday, or on September 11th every year. This will be different from your original plan. So, you have to pay attention to this point! The above writing is incorrect!

View crontab execution history under CentOS

I added a scheduled task to crontab, but did not get the expected result. I suspected that crontab did not execute the corresponding task. But how can I determine whether crontab is executed?

This requires viewing the execution history of crontab, the specific location is as follows:

cd /var/log
tail -100 cron

You can check the related scheduled tasks that have been operated in the cron file.

Crontab command format description

We can use crontab -e to add commands to be executed. The result of command execution, whether standard output or error output, will be sent to the user via email.

The added command must be in the following format:

 * * * * * /command path

The first five fields can take integer values ​​to specify when to start the work, and the sixth field is a string, that is, the command field, which includes the command scheduled to be executed by crontab. Each field is separated by spaces and tabs.

The first five fields represent:

Minutes: 0-59
Hours: 1-23
Date: 1-31
Month: 1-12
Day of the week: 0-6 (0 means Sunday)

Some special symbols can also be used:

*: Any time
,: indicates segmentation -: indicates a segment, such as the second end: 1-5, which means 1 to 5 points
/n: means executing once for every n units. For example, in the second paragraph, */1 means executing the command once every hour. It can also be written as 1-23/1.

Some examples:

00 8,12,16 * * * /data/app/scripts/monitor/df.sh
30 2 * * * /data/app/scripts/hotbackup/hot_database_backup.sh
10 8,12,16 * * * /data/app/scripts/monitor/check_ind_unusable.sh
10 8,12,16 * * * /data/app/scripts/monitor/check_maxfilesize.sh
10 8,12,16 * * * /data/app/scripts/monitor/check_objectsize.sh

: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

Crontab command background execution &

When a job is running in the foreground, the terminal is occupied by the job; when a job is running in the background, it does not occupy the terminal. You can use the & command to put the job into the background for execution.

like:

30 2 * * * /data/app/scripts/hotbackup/hot_database_backup.sh &

Be careful when running jobs in the background: do not run commands that require user interaction in the background, because then your machine will just sit there waiting.

However, jobs running in the background will still output results to the screen, interfering with your work. If the job you want to run in the background generates a lot of output, it is best to redirect its output to a file using the following method:

like:

command >out.file 2>&1 &

In this example, 2>&1 means that all standard output and error output will be redirected to a file called out.file.

Crontab command 2>&1 meaning

Let’s look at an example:

0 2 * * * /u01/test.sh >/dev/null 2>&1 &

This sentence means to execute this command in the background, redirect error output 2 to standard output 1, and then put all standard output 1 into the /dev/null file, that is, clear it.

Here are a few numbers that mean:

0 means keyboard input
1 means standard output
2 indicates error output

We could also write:

0 2 * * * /u01/test.sh 1>/u01/out.file &
0 2 * * * /u01/test.sh 2>/u01/out.file &
0 2 * * * /u01/test.sh 2>/u01/out.file 2>&1 &

Redirect the tesh.sh command output to out.file, that is, the output content is not printed on the screen, but output to the out.file file.

2>&1 redirects error output to standard output. Then redirect standard input to the file out.file.

&1 represents file description 1, which means standard output. If & is missing here, it becomes the number 1, which means redirection to file 1.

& ：Background execution

test:

ls 2>1: It will not report an error about the absence of file 2, but will output an empty file 1;
ls xxx 2>1: The error that there is no file xxx is output to 1;
ls xxx 2>&1: The file 1 will not be generated, but the error will go to standard output;
ls xxx >out.txt 2>&1 == ls xxx 1>out.txt 2>&1: Because the redirection symbol > defaults to 1, this sentence will transfer both the error output and the standard output to the out.txt file.

The reason why the Crontab command 2>&1 is written at the end

Format: command > file 2>&1 == command 1> file 2>&1

First, command > file redirects the standard output to file, and 2>&1 is the standard error copying the standard output, which is also redirected to file. The final result is that both the standard output and the error are redirected to file.

If changed to: command 2>&1 >file

2>&1 Standard error copies the behavior of standard output, but standard output is still in the terminal. >file The output is redirected to file, but the standard error remains in the terminal.

Lessons learned:

I plan to execute a Python script on the server at 23:00 every night to back up the MySql database. The command is as follows:

* 23 * * * python /var/www/html/crontab_python/back_db.py >/dev/null 2>&1

As a result, 60 backup files were generated each time. After carefully checking the scheduled task command, I found that a "0" was missing at the "minute" position, because "*" represents any value at that position. The modification is as follows:

0 23 * * * python /var/www/html/crontab_python/back_db.py >/dev/null 2>&1

Then the scheduled execution of the PHP script

0 4 * * * /usr/local/php/bin/php /usr/local/nginx/www/backup-db/backup_db.php 172.16.8.26 >/dev/null 2>&1
0 4 * * * /usr/local/php/bin/php /usr/local/nginx/www/backup-db/backup_db.php 172.16.10.151 >/dev/null 2>&1

This article introduces the detailed usage of the Linux scheduled task Crontab command and summarizes some usage tips of the Crontab command and some solutions to problems encountered in work. I hope it will be helpful to everyone