一、简介
awk 是一个处理文本的编程语言工具，能用简短的程序处理标准输入或文件、数据排序、计算以及 生成报表等等。
在 Linux 系统下默认 awk 是 gawk，它是 awk 的 GNU 版本。可以通过命令查看应用的版本：ls -l /bin/awk
基本的命令语法： awk option 'pattern {action}' file
其中 pattern 表示 AWK 在数据中查找的内容，而 action 是在找到匹配内容时所执行的一系列命令。 花括号用于根据特定的模式对一系列指令进行分组。
awk 处理的工作方式与数据库类似，支持对记录和字段处理，这也是 grep 和 sed 不能实现的。 在 awk 中，缺省的情况下将文本文件中的一行视为一个记录，逐行放到内存中处理，而将一行中的 某一部分作为记录中的一个字段。用 1,2,3...数字的方式顺序的表示行（记录）中的不同字段。用 $后跟数字，引用对应的字段，以逗号分隔，0 表示整个行。
二、OPTION
选项描述
| 选项 | 描述 |
|---|---|
| -f | 从文件中读取 awk 程序源文件 |
| -F | 指定 fs 为输入字段分隔符 |
| -v | var=value 变量赋值 |
| --posix | 兼容 POSIX 正则表达式 |
| --dump-variables=[file] | 把 awk 命令时的全局变量写入文件， 默认文件是 awkvars.out |
三、pattern
模式描述
| 模式 | 描述 |
|---|---|
| BEGIN{ } | 给程序赋予初始状态，先执行的工作 |
| END{ } | 程序结束之后执行的一些扫尾工作 |
| /regular expression/ | 为每个输入记录匹配正则表达式 |
| pattern && pattern | 逻辑 and，满足两个模式 |
| pattern1, pattern2 | 范围模式，匹配所有模式 1 的记录，直到匹配到模式 2 |
| action | 就是print、流程控制。I/O语句 |
四、案例
1.从文件读取awk的程序
[root@localhost ~]# cat test.awk
{print $2}
[root@localhost ~]# tail -n3 /etc/services |awk -f test.awk
48619/tcp
48619/udp
49000/tcp
2.变量赋值
[root@192 ~]# awk -v a=1 'BEGIN{print a}'
1
[root@192 ~]# a=3
[root@192 ~]# awk -v a=$a 'BEGIN{print a}'
3
[root@192 ~]# awk 'BEGIN{print '$a'}' 单引号调用变量
3
3.全局变量到文件
[root@localhost ~]# seq 5|awk --dump-variables '{print $0}'
1
2
3
4
5
[root@localhost ~]# cat awkvars.out 默认文件
ARGC: 1
ARGIND: 0
ARGV: array, 1 elements
BINMODE: 0
CONVFMT: "%.6g"
ERRNO: ""
FIELDWIDTHS: ""
FILENAME: "-"
FNR: 5
FPAT: "[^[:space:]]+"
FS: " "
IGNORECASE: 0
LINT: 0
NF: 1
NR: 5
OFMT: "%.6g"
OFS: " "
ORS: "
"
RLENGTH: 0
RS: "
"
RSTART: 0
RT: "
"
SUBSEP: " 34"
TEXTDOMAIN: "messages"
4.BEGIN 处理文件前的操作，修改内置变量 变量赋值 页眉和标题
[root@192 ~]# tail /etc/services |awk 'BEGIN{print "Service Tport Description"}{print $0}'
Service Tport Description
3gpp-cbsp 48049/tcp # 3GPP Cell Broadcast Service Protocol
isnetserv 48128/tcp # Image Systems Network Services
isnetserv 48128/udp # Image Systems Network Services
blp5 48129/tcp # Bloomberg locator
blp5 48129/udp # Bloomberg locator
com-bardac-dw 48556/tcp # com-bardac-dw
com-bardac-dw 48556/udp # com-bardac-dw
iqobject 48619/tcp # iqobject
iqobject 48619/udp # iqobject
matahari 49000/tcp # Matahari Broker
5.END 程序执行完了 才会执行后面的END的动作
[root@192 ~]# tail /etc/services |awk '{print $0}END{print "====
END"}'
3gpp-cbsp 48049/tcp # 3GPP Cell Broadcast Service Protocol
isnetserv 48128/tcp # Image Systems Network Services
isnetserv 48128/udp # Image Systems Network Services
blp5 48129/tcp # Bloomberg locator
blp5 48129/udp # Bloomberg locator
com-bardac-dw 48556/tcp # com-bardac-dw
com-bardac-dw 48556/udp # com-bardac-dw
iqobject 48619/tcp # iqobject
iqobject 48619/udp # iqobject
matahari 49000/tcp # Matahari Broker
====
END
6. --profile 输出命令到文件
[root@192 ~]# tail /etc/services |awk --profile 'BEGIN{print "service tport de
=="}{print $0}END{print"====
END...."}'
service tport de
==
3gpp-cbsp 48049/tcp # 3GPP Cell Broadcast Service Protocol
isnetserv 48128/tcp # Image Systems Network Services
isnetserv 48128/udp # Image Systems Network Services
blp5 48129/tcp # Bloomberg locator
blp5 48129/udp # Bloomberg locator
com-bardac-dw 48556/tcp # com-bardac-dw
com-bardac-dw 48556/udp # com-bardac-dw
iqobject 48619/tcp # iqobject
iqobject 48619/udp # iqobject
matahari 49000/tcp # Matahari Broker
====
END....
[root@localhost ~]# cat awkprof.out
# gawk profile, created Mon Jul 22 17:13:27 2019
# BEGIN block(s)
BEGIN {
print "service tport de
=="
}
# Rule(s)
{
print $0
}
# END block(s)
END {
print "====
END...."
}
7./re/ 正则匹配
匹配第一个字段是8个字符行的
[root@192 ~]# tail /etc/services |awk '/^[a-z0-9]{8}/{print $0}'
isnetserv 48128/tcp # Image Systems Network Services
isnetserv 48128/udp # Image Systems Network Services
iqobject 48619/tcp # iqobject
iqobject 48619/udp # iqobject
matahari 49000/tcp # Matahari Broker
匹配tcp字段的行
[root@192 ~]# tail /etc/services |awk '/tcp/{print $0}'
3gpp-cbsp 48049/tcp # 3GPP Cell Broadcast Service Protocol
isnetserv 48128/tcp # Image Systems Network Services
blp5 48129/tcp # Bloomberg locator
com-bardac-dw 48556/tcp # com-bardac-dw
iqobject 48619/tcp # iqobject
matahari 49000/tcp # Matahari Broker
匹配已blp5开头的行
[root@192 ~]# tail /etc/services |awk '/^blp5/{print $0}'
blp5 48129/tcp # Bloomberg locator
blp5 48129/udp # Bloomberg locator
8. 逻辑and or not
匹配blp5和tcp的行
[root@192 ~]# tail /etc/services |awk '/blp5/ && /tcp/{print $0}'
blp5 48129/tcp # Bloomberg locator
匹配blp5或tcp的行
[root@192 ~]# tail /etc/services |awk '/blp5/ | /tcp/{print $0}'
3gpp-cbsp 48049/tcp # 3GPP Cell Broadcast Service Protocol
isnetserv 48128/tcp # Image Systems Network Services
blp5 48129/tcp # Bloomberg locator
blp5 48129/udp # Bloomberg locator
com-bardac-dw 48556/tcp # com-bardac-dw
iqobject 48619/tcp # iqobject
matahari 49000/tcp # Matahari Broker
或
[root@192 ~]# tail /etc/services |awk '/blp5|tcp/{print $0}'
3gpp-cbsp 48049/tcp # 3GPP Cell Broadcast Service Protocol
isnetserv 48128/tcp # Image Systems Network Services
blp5 48129/tcp # Bloomberg locator
blp5 48129/udp # Bloomberg locator
com-bardac-dw 48556/tcp # com-bardac-dw
iqobject 48619/tcp # iqobject
matahari 49000/tcp # Matahari Broker
不匹配#开头和空行
[root@192 ~]# cat test.txt|awk '!/^#/ && !/^$/{print $0}'
dadadad
111111
[root@192 ~]# cat test.txt
#dasdsdadad
dadadad
111111
#vvvvv
或
[root@192 ~]# cat test.txt |awk '/^[^#]|"^$"/' _待解释_
dadadad
111111
9.匹配范围
开始匹配第一个关键字，到匹配第二个关键字结束
[root@192 ~]# tail /etc/services |awk '/^3gp/,/^com/{print $0}'
3gpp-cbsp 48049/tcp # 3GPP Cell Broadcast Service Protocol
isnetserv 48128/tcp # Image Systems Network Services
isnetserv 48128/udp # Image Systems Network Services
blp5 48129/tcp # Bloomberg locator
blp5 48129/udp # Bloomberg locator
com-bardac-dw 48556/tcp # com-bardac-dw