Common shell script commands and related knowledge under Linux

1. Some points to remember

1. Variable Type

When running the shell, three variables exist at the same time:

1) Local variables

Local variables are defined in scripts or commands and are valid only in the current shell instance. Programs started by other shells cannot access local variables.

2) Environment variables

All programs, including those started by the shell, can access environment variables, and some programs require environment variables to ensure their normal operation. Shell scripts can also define environment variables when necessary.

3) Shell variables

Shell variables are special variables set by the shell program. Some shell variables are environment variables, and some are local variables. These variables ensure the normal operation of the shell.

2. Shell variable description

$$
The PID (ProcessID) of the shell itself
$!
PID of the background process last run by the Shell
$?
The exit code (return value) of the last command executed
$-
List of Flags set using the Set command
$*
List of all parameters. If "$*" is enclosed in "", all parameters are output in the form of "$1 $2 ... $n".
$@
List of all parameters. If "$@" is enclosed in "" brackets, all parameters are output in the form of "$1" "$2" … "$n".
$#
The number of parameters added to the Shell
$0
The file name of the shell itself
$1～$n
Add the parameter values ​​to the Shell. $1 is the first parameter, $2 is the second parameter, and so on.

3. The difference between single quotes, double quotes and back quotes

When single quotes ' ' are used to enclose the value of a variable, what is inside the single quotes will be output, even if there are variables and commands (commands need to be enclosed in quotes) in the content, they will be output as is.
When double quotes " " are used to enclose the value of a variable, the variables and commands inside will be parsed first when outputting, rather than outputting the variable names and commands in the double quotes as is.

With backticks ` ` (below the esc key), command substitution means that the shell can execute the command first, save the output temporarily, and then output it in the appropriate place. You can specify variables using

4. Shell escape characters

\\ Backslash
\a Alarm, ringing
\b Backspace (Delete key)
\f Page feed (FF), move the current position to the beginning of the next page
\n Line break
\r Enter
\t Horizontal tab character (tab key)
\v vertical tab character

5. List of arithmetic operators

Operator Description Example
+ The addition `expr $a + $b` results in 30.
- The result of the subtraction `expr $a - $b` is 10.
\* The multiplication `expr $a \* $b` results in 200.
/ The division `expr $b / $a` results in 2.
% The remainder of `expr $b % $a` is 0.
= Assignment a=$b will assign the value of variable b to a.
== is equal. Compares two numbers and returns true if they are the same. [ $a == $b ] returns false.
!= Not equal. Compares two numbers and returns true if they are different. [ $a != $b ] returns true.

6. Relational operators

Relational operators only support numbers, not strings, unless the string value is a number.

Operator Description Example
-eq checks whether two numbers are equal, and returns true if they are equal. [ $a -eq $b ] returns true.
-ne checks whether two numbers are equal, and returns true if they are not equal. [ $a -ne $b ] returns true.
-gt checks whether the number on the left is greater than the number on the right, and returns true if it is. [ $a -gt $b ] returns false.
-lt Checks whether the number on the left is less than the number on the right, and returns true if it is. [ $a -lt $b ] returns true.
-ge checks whether the number on the left is greater than or equal to the number on the right, and returns true if it is. [ $a -ge $b ] returns false.
-le checks whether the number on the left is less than or equal to the number on the right, and returns true if it is. [ $a -le $b ] returns true.

7. List of Boolean Operators

Operator Description Example
! Not an operation, if the expression is true, it returns false, otherwise it returns true. [ ! false ] returns true.
-o OR operation, returns true if one expression is true. [ $a -lt 20 -o $b -gt 100 ] returns true.
-a AND operation, returns true only if both expressions are true. [ $a -lt 20 -a $b -gt 100 ] returns false.

8. List of file test operators

Operator Description Example
-b file Checks whether file is a block device file and returns true if it is. [ -b $file ] returns false.
-c file Checks whether file is a character device file and returns true if it is. [ -b $file ] returns false.
-d file Checks whether file is a directory and returns true if it is. [ -d $file ] returns false.
-f file Checks whether file is a normal file (neither a directory nor a device file) and returns true if it is. [ -f $file ] returns true.
-g file Checks whether the SGID bit is set on the file and returns true if so. [ -g $file ] returns false.
-k file Checks whether the file has the sticky bit set, and returns true if so. [ -k $file ] returns false.
-p file Checks whether file is a named pipe and returns true if it is. [ -p $file ] returns false.
-u file Checks whether the SUID bit is set on a file and returns true if so. [ -u $file ] returns false.
-r file Checks whether the file is readable and returns true if it is. [ -r $file ] returns true.
-w file Checks whether the file is writable and returns true if it is. [ -w $file ] returns true.
-x file Checks whether the file is executable and returns true if it is. [ -x $file ] returns true.
-s file checks whether the file is empty (whether the file size is greater than 0), and returns true if it is not empty. [ -s $file ] returns true.
-e file Checks whether a file (including directories) exists and returns true if it does. [ -e $file ] returns true.

2. Examples of commonly used script commands

The example is too long, so I will only list one.

1. We store nginx logs in nginx.log, count the number of access IPs on April 23, 2020, and sort them in descending order

Example: 192.168.1.20 - - [21/Apr/2020:14:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"

cat nginx.log | grep 23/Apr/2020 | awk -F "-" '{print $1}'|sort|uniq -c | sort -r | awk '{print $1,$2}'

2. We store nginx logs in nginx.log and count the number of duplicate IP visits from 20:00 to 23:00 on April 23, 2020

Example: 192.168.1.20 - - [21/Apr/2020:14:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"

cat nginx.log | grep 23/Apr/2020:2[0-3] | awk '{print $1}' | sort | uniq | wc -l

3. We store nginx logs in nginx.log and write scripts to count IP addresses that have visited more than 3 times

Example: 192.168.1.20 - - [21/Apr/2020:14:27:49 +0800] "GET /1/index.php HTTP/1.1" 404 490 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:45.0) Gecko/20100101 Firefox/45.0"

cat nginx.log | awk '{print $1}'| sort | uniq -c | awk '{if ($1 >3) print $0}' | sort -r | awk '{print $1,$2}'

awk data field variables
$0 represents the entire line of text
$1 represents the first data field in the text
$2 represents the second data field in the text
$n represents the nth data field in the text

4. Put the results of the netstat command in netstat.txt, view all IPs that have established a connection with port 3306 of the local machine and are in the established state, and sort them in descending order according to the number of connections

Example: tcp 0 0 172.16.56.200:41856 172.16.34.144:3306 ESTABLISHED

cat netstat.txt | grep ESTABLISHED | grep 3306 | awk '{print $5}' | awk -F ":" '{print $1}' | sort | uniq -c | sort -hr | awk '{print $1,$2}'

5. Count the used or unused IP addresses in the network segment

#!/bin/bash
for ip in `seq 0 255`
 do
 ping -c 1 -i 0 192.168.2.$ip
if [ $? -eq 0 ]
then
 echo "192.168.2.$ip" >> /root/up.txt
else
 echo "192.168.2.$ip" >> /root/down.txt
fi
done

6. Read the number of lines in the file and read them line by line in a loop

Example: cat http

www.baidu.com

www.cityhouse.cn

www.cityre.cn

vim htttp.sh

cat /data/script/http | while read line
do
  curl $line 
done
date=`date "+%Y-%m-%d-%H-%M-%S"`
echo "sucessful$date" >> /data/script/http.txt

This is the end of this article about commonly used shell script commands and related knowledge under Linux. For more relevant shell script commands under Linux, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope everyone will support 123WORDPRESS.COM in the future!