Detailed explanation of Linx awk introductory tutorial

Awk is an application for processing text files, and almost all Linux systems come with this program.

It processes each line of the file in turn and reads each field in it. For text files like logs and CSV where each line has the same format, awk may be the most convenient tool.

Awk is not only a tool software, but also a programming language. However, this article only introduces its command line usage, which should be sufficient for most occasions.

1. Basic Usage

The basic usage of awk is in the following form.

# Format $awk action file name # Example $awk '{print $0}' demo.txt

In the above example, demo.txt is the text file to be processed by awk. There is a curly brace inside the single quotes, which contains the processing action print $0 for each line. Among them, print is the print command, $0 represents the current line, so the execution result of the above command is to print out each line as it is.

Next, we will use standard input (stdin) to demonstrate the above example.

$ echo 'this is a test' | awk '{print $0}'
this is a test

In the above code, print $0 is to reprint the standard input this is a test.

Awk will divide each line into several fields according to spaces and tabs, using $1, $2, $3 to represent the first field, the second field, the third field, and so on.

$ echo 'this is a test' | awk '{print $3}'
a

In the above code, $3 represents the third field a of this is a test.

Below, for the sake of example, we save the /etc/passwd file as demo.txt.

root:x:0:0:root:/root:/usr/bin/zsh
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin
sys:x:3:3:sys:/dev:/usr/sbin/nologin
sync:x:4:65534:sync:/bin:/bin/sync

The field separator of this file is a colon (:), so you need to use the -F parameter to specify the separator as a colon. Then, its first field can be extracted.

$ awk -F ':' '{ print $1 }' demo.txt
root
daemon
bin
sys
sync

grammar

awk [option parameter] 'script' var=value file(s)

or

awk [option parameter] -f scriptfile var=value file(s)

Variables

In addition to $ + number to represent a field, awk also provides some other variables.

The variable NF indicates how many fields there are in the current line, so $NF represents the last field.

$ echo 'this is a test' | awk '{print $NF}'
test
$(NF-1) represents the second to last field.
$ awk -F ':' '{print $1, $(NF-1)}' demo.txt
root /root
daemon /usr/sbin
bin /bin
sys /dev
sync /bin

In the above code, the comma in the print command indicates that a space is used to separate the two parts during output.
The variable NR indicates which row is currently being processed.

$ awk -F ':' '{print NR ") " $1}' demo.txt

1) root
2) daemon
3) bin
4) sys
5) sync

In the above code, in the print command, if you want to output characters as they are, you need to put them in double quotes.

Other built-in variables of awk are as follows.

FILENAME: Current file name
FS: Field separator, the default is space and tab.
RS: Line separator, used to separate each line. The default is newline.
OFS: Output field separator, used to separate fields when printing, the default is space.
ORS: The output record separator, used to separate records when printing. The default is a newline character.
OFMT: The format of digital output, the default is %.6g

3. Function

Awk also provides some built-in functions to facilitate the processing of raw data.

The function toupper() is used to convert characters to uppercase.

$ awk -F ':' '{ print toupper($1) }' demo.txt
ROOT
DAEMON
BIN
SYS
SYNC

In the above code, the first field is converted to uppercase when output.

Other commonly used functions are as follows.

tolower(): Convert characters to lowercase.
length(): Returns the length of a string.
substr(): Returns a substring.
sin(): sine.
cos(): Cosine.
sqrt(): Square root.
rand(): Random number.

For a complete list of awk's built-in functions, see the manual page.

IV. Conditions

Awk allows you to specify output conditions and only output lines that meet the conditions.

The output condition should be written before the action.

$ awk 'conditional action' file name

Please see the example below.

$ awk -F ':' '/usr/ {print $1}' demo.txt
root
daemon
bin
sys

In the above code, the print command is preceded by a regular expression, which only outputs the lines containing usr.

The following example outputs only odd-numbered lines, and only the lines after the third line.

# Output odd lines $ awk -F ':' 'NR % 2 == 1 {print $1}' demo.txt
root
bin
sync

# Output the third and subsequent lines $ awk -F ':' 'NR >3 {print $1}' demo.txt
sys
sync

The following example outputs the row where the first field is equal to the specified value.

$ awk -F ':' '$1 == "root" {print $1}' demo.txt
root

$ awk -F ':' '$1 == "root" || $1 == "bin" {print $1}' demo.txt
root
bin

5. if statement

Awk provides the if structure for writing complex conditions.

$ awk -F ':' '{if ($1 > "m") print $1}' demo.txt
root
sys
sync

The above code outputs the lines where the first character of the first field is greater than m.

The if structure can also specify an else part.

$ awk -F ':' '{if ($1 > "m") print $1; else print "---"}' demo.txt
root
---
---
sys
sync

6. Reference Links

An Awk tutorial by Example, Greg Grothaus

30 Examples for Awk Command in Text Processing, Mokhtar Ebrahim

Summarize

The above is the Linx awk introductory tutorial introduced by the editor. I hope it will be helpful to everyone. If you have any questions, please leave me a message and the editor will reply to you in time. I would also like to thank everyone for their support of the 123WORDPRESS.COM website!