Sed Awk 日常使用总结

Sed命令语法
sed [option] {sed-commands}{input-file}
sed首先从input-file中读取第一行，然后执行所有的sed-commands；再读取第二行，执行所有sed-commands,
重复这个过程，直到input-file结束。
sed的执行过程：读取内容，执行命令，打印结果，重复循环。

打印模式空间(命令p)
$-代表最后一行
修改地址范围：逗号，加号，波浪号。
1,4-第一到第四行
2,$-第二到最后一行
n+m-从第n行开始后的m行
1~2匹配1,3,5,7,......
2~2匹配2,4,6,......
1~3匹配1,4,7,......
2~3匹配2,5,8,11,......
[root@node130 ~]# sed 'p' ./employee.txt
101,John Doe,CEO
101,John Doe,CEO
102,Jason Smith,IT Manager
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
104,Anand Ram,Developer
105,jane Miller,Sales Manager
105,jane Miller,Sales Manager
[root@node130 ~]# vim employee.txt
[root@node130 ~]# sed 'p' employee.txt
101,John Doe,CEO
101,John Doe,CEO
102,Jason Smith,IT Manager
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
104,Anand Ram,Developer
105,jane Miller,Sales Manager
105,jane Miller,Sales Manager
[root@node130 ~]# sed -n 'p' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,jane Miller,Sales Manager
[root@node130 ~]# sed -n  employee.txt
^C
[root@node130 ~]# sed -n 'p' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,jane Miller,Sales Manager
[root@node130 ~]# sed -n '2p' employee.txt
102,Jason Smith,IT Manager
[root@node130 ~]# sed -n '2 p' employee.txt
102,Jason Smith,IT Manager
[root@node130 ~]# sed -n '1,2 p' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
[root@node130 ~]# sed -n '1,24 p' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,jane Miller,Sales Manager
[root@node130 ~]# sed -n '1,4p' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
[root@node130 ~]# sed -n '1,4 p' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
[root@node130 ~]# sed -n '2,$ p' employee.txt
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,jane Miller,Sales Manager
[root@node130 ~]# sed -n '2,+2 p' employee.txt
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
[root@node130 ~]# sed -n '1~2 p' employee.txt
101,John Doe,CEO
103,Raj Reddy,Sysadmin
105,jane Miller,Sales Manager
[root@node130 ~]# sed -n '2~2 p' employee.txt
102,Jason Smith,IT Manager
104,Anand Ram,Developer
[root@node130 ~]# sed -n '1~3 p' employee.txt
101,John Doe,CEO
104,Anand Ram,Developer
[root@node130 ~]# sed -n '/Jane/p' employee.txt
[root@node130 ~]# sed -n '/Jane/ p' employee.txt
[root@node130 ~]# cat employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,jane Miller,Sales Manager
[root@node130 ~]# vim employee.txt
[root@node130 ~]# sed -n '/Jane/p' employee.txt
105,Jane Miller,Sales Manager
[root@node130 ~]# sed -n '/Jason/,4p' employee.txt
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
[root@node130 ~]# sed -n '/Raj/,$ p' employee.txt
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 ~]# sed -n '/Raj/,/Jane/ p' employee.txt
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 ~]# sed -n '/Jason,+2 p' employee.txt
sed: -e expression #1, char 11: unterminated address regex
[root@node130 ~]# sed -n '/Jane,+2 p' employee.txt
sed: -e expression #1, char 10: unterminated address regex
[root@node130 ~]# sed -n '/Jane,+2 p' employee.txt
sed: -e expression #1, char 10: unterminated address regex
删除行
命令d用来删除行;如果不提供地址范围，sed默认匹配所有行，但不修改原始文件的内容。
[root@node130 ~]# sed 'd' employee.txt
[root@node130 ~]# cat employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 ~]# sed '2d' employee.txt
101,John Doe,CEO
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 ~]# sed '2,3d' employee.txt
101,John Doe,CEO
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 ~]# sed '2,4d' employee.txt
101,John Doe,CEO
105,Jane Miller,Sales Manager
[root@node130 ~]# sed '1,4d' employee.txt
105,Jane Miller,Sales Manager
[root@node130 ~]# sed '2,$d' employee.txt
101,John Doe,CEO
把模式空间内容写到文件中(w命令)
命令w可以把当前模式空间内容保存到文件中。默认情况下模式空间的内容每次都会
打印到标准输出，如果要把输出保存到文件同时不显示到屏幕上，还需要使用—n选项
[root@node130 ~]# ls
anaconda-ks.cfg  Documents  employee.txt  hosts2       install.log.syslog                                 Music     Public     test             tutor
Desktop          Downloads  hosts         install.log  KingbaseAnalyticsDB-3.0-build1-CENTOS6-x86_64.run  Pictures  Templates  test-script.sed  Videos
[root@node130 ~]# sed 'w output.txt' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 ~]# ls 
anaconda-ks.cfg  Documents  employee.txt  hosts2       install.log.syslog                                 Music       Pictures  Templates  test-script.sed  Videos
Desktop          Downloads  hosts         install.log  KingbaseAnalyticsDB-3.0-build1-CENTOS6-x86_64.run  output.txt  Public    test       tutor
[root@node130 ~]# vim output.txt
[root@node130 ~]# cat output.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 ~]# sed 'w output000.txt' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 ~]# sed -n 'w output007.ext' employee.txt
[root@node130 ~]# cat output00
output000.txt  output007.ext 
[root@node130 ~]# cat output007.ext
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 ~]# sed -n '2 w output2_line.txt' employee.txt
[root@node130 ~]# cat output2_line.txt
102,Jason Smith,IT Manager
[root@node130 ~]# sed -n '1,4 w output_1_4.txt' employee.txt
[root@node130 ~]# cat output_1_4.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
[root@node130 ~]# sed -n '2,$ w output_2_$.txt' employee.txt
[root@node130 ~]# cat output_2_$.txt
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 ~]# sed -n '1~2 w output_1~2.txt' employee.txt
[root@node130 ~]# ls
anaconda-ks.cfg  Downloads     hosts2              KingbaseAnalyticsDB-3.0-build1-CENTOS6-x86_64.run  output007.ext   output2_line.txt  Pictures   test             Videos
Desktop          employee.txt  install.log         Music                                              output_1~2.txt  output_2_$.txt    Public     test-script.sed
Documents        hosts         install.log.syslog  output000.txt                                      output_1_4.txt  output.txt        Templates  tutor
[root@node130 ~]# cat output_1~2.txt
101,John Doe,CEO
103,Raj Reddy,Sysadmin
105,Jane Miller,Sales Manager
[root@node130 ~]# sed -n '/Jane/ w output.Jane.txt' employee.txt
[root@node130 ~]# vim output.Jane.txt
[root@node130 ~]# vim output.Jane.txt
[root@node130 ~]# sed -n '/Jason/,4 w output.Jason.4.txt' employee.txt
[root@node130 ~]# vim output.Jason.4.txt
[root@node130 ~]# sed -n '/Raj/,$ w output.Raj.$.txt' employee.txt
[root@node130 ~]# vim output.Raj.$.txt
[root@node130 ~]# sed -n '/Raj/,/Jane/ w output.Raj.Fane' employee.txt
[root@node130 ~]# vim output.Raj.Fane
[root@node130 ~]# sed -n '/Jason/,+2 w output.Jason.2.txt' employee.txt
[root@node130 ~]# vim output.Ja
output.Jane.txt     output.Jason.2.txt  output.Jason.4.txt 
[root@node130 ~]# vim output.Ja
output.Jane.txt     output.Jason.2.txt  output.Jason.4.txt 
[root@node130 ~]# vim output.Jason.2.txt
sed替换命令
sed中最强大的功能就是替换(substitute),功能强大。

sed '[address-range|pattern-range] s/original-string/replacement-string/[substitute-flags]' input-file
s-即执行替换命令substitute
original-string-是被sed搜索然后被替换的字符串，可以是一个正则表达式
replacement-string-替换后的字符串
substitute-flags-可选的，全局标志
全局标志g
g代表全局(global)默认情况下，sed只会替换每行中第一次出现的original-string;如果要替换每行中出现的
所有original-string，就需要使用g.
数字标志(1,2,3,......,512)
使用数字可以指定original-string出现的次序。只有第n次出现的original-string才会触发替换。
打印标志p(print)
命令p代表print。当替换操作完成后，打印替换后的行；
sed中比较有用的方法是和-n一起使用以抑制默认的打印操作。
写标志w
标志w代表write,当替换成功后，它把替换后的结果保存在文件中。
忽略大小写标志i(ignore)
替换标识i代表忽略大小写。可以使用i来以小写字符的模式匹配original-string。只有GNU sed中才有。
执行命令标志e(excute)
替换标志e代表执行(execute).该标志可以将模式空间中的任何内容当作shell命令执行，并把命令执行的结果
返回到模式空间。只有GNU sed中才可使用。
[root@node130 ~]# sed 's/Manager/Director/' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Director
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Director
[root@node130 ~]# sed 's/Manager/Director/' employee.txt ;cat employee.txt
101,John Doe,CEO
102,Jason Smith,IT Director
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Director
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 ~]# sed '/Sales/s/Manager/Director' employee.txt
sed: -e expression #1, char 25: unterminated `s' command
[root@node130 ~]# sed '/Sales/s/Manager/Director/' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Director
[root@node130 ~]# sed '/s/a/A' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
/A
103,Raj Reddy,Sysadmin
/A
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
/A
[root@node130 ~]# sed 's/a/A' employee.txt
sed: -e expression #1, char 5: unterminated `s' command
[root@node130 ~]# sed 's/a/A/' employee.txt
101,John Doe,CEO
102,JAson Smith,IT Manager
103,RAj Reddy,Sysadmin
104,AnAnd Ram,Developer
105,JAne Miller,Sales Manager
[root@node130 ~]# sed 's/a/A/g' employee.txt
101,John Doe,CEO
102,JAson Smith,IT MAnAger
103,RAj Reddy,SysAdmin
104,AnAnd RAm,Developer
105,JAne Miller,SAles MAnAger
[root@node130 ~]# sed 's/a/A/2' employee.txt
101,John Doe,CEO
102,Jason Smith,IT MAnager
103,Raj Reddy,SysAdmin
104,Anand RAm,Developer
105,Jane Miller,SAles Manager
[root@node130 ~]# vim substitute-locate.txt
[root@node130 ~]# sed 's/locate/find/2' substitute-locate.txt
locate command is used to find files
locate command uses database to find files
locate command can also use regex for searching
[root@node130 ~]# sed -n 's/John/Johnny' employee.txt
sed: -e expression #1, char 13: unterminated `s' command
[root@node130 ~]# sed -n 's/John/Johnny/' employee.txt
[root@node130 ~]# sed  's/John/Johnny/' employee.txt
101,Johnny Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 ~]# sed -n 's/John/Johnny/p' employee.txt
101,Johnny Doe,CEO
[root@node130 ~]# sed -n 's/locate/find/2p' substitute-locate.txt
locate command is used to find files
locate command uses database to find files
[root@node130 ~]# sed -n 's/John/Johnny/w Johnny.txt' employee.txt
[root@node130 ~]# vim Johnny.txt
[root@node130 ~]# sed 's/locate/find/2w locate_find.txt' substitute-locate.txt
locate command is used to find files
locate command uses database to find files
locate command can also use regex for searching
[root@node130 ~]# vim locate_find.txt
[root@node130 ~]# vim locate_find.txt
[root@node130 ~]# sed 's/john/Johnny/' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 ~]# sed 's/john/Johnny/i' employee.txt
101,Johnny Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 ~]# ls
anaconda-ks.cfg  hosts               KingbaseAnalyticsDB-3.0-build1-CENTOS6-x86_64.run  output_1~2.txt    output.Jason.2.txt  Pictures               test-script.sed
Desktop          hosts2              locate_find.txt                                    output_1_4.txt    output.Jason.4.txt  Public                 tutor
Documents        install.log         Music                                              output2_line.txt  output.Raj.Fane     substitute-locate.txt  Videos
Downloads        install.log.syslog  output000.txt                                      output_2_$.txt    output.Raj.$.txt    Templates
employee.txt     Johnny.txt          output007.ext                                      output.Jane.txt   output.txt          test
[root@node130 ~]# mkdir sed
[root@node130 ~]# cd sed/
[root@node130 sed]# ls
[root@node130 sed]# touch files.txt
[root@node130 sed]# ;s
bash: syntax error near unexpected token `;'
[root@node130 sed]# ls
files.txt
[root@node130 sed]# cp /etc/passwd .
[root@node130 sed]# ls
files.txt  passwd
[root@node130 sed]# cp /etc/gr
group      group-     grub.conf 
[root@node130 sed]# cp /etc/group .
[root@node130 sed]# ls
files.txt  group  passwd
[root@node130 sed]# vim group
[root@node130 sed]# sed 's/^/ls -l/' files.txt
[root@node130 sed]# vim files.txt
[root@node130 sed]# ls -l /etc/passwd
-rw-r--r--. 1 root root 1703 Jan 10 02:20 /etc/passwd
[root@node130 sed]# sed 's/^/ls -l/e' files.txt
[root@node130 sed]# sed -n 's/manager/Director/igpw output.txt' employee.txt
102,Jason Smith,IT Director
105,Jane Miller,Sales Director
[root@node130 sed]# cat output.txt
102,Jason Smith,IT Director
105,Jane Miller,Sales Director
[root@node130 sed]# vim path.txt
sed替换命令分界符
sed默认分界符/,即s/original-string/replacement-string/g
如果在original-string或replacement-string中有/,那么需要使用反斜杠来转义。
可以使用任何一个字符作为sed替换命令的分节符，如|或^或@或!。
单行内容上执行多个命令
[root@node130 sed]# sed '{
> s/Developer/IT Manager/
> s/Manager/Director/
> }' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Director
103,Raj Reddy,Sysadmin
104,Anand Ram,IT Director
105,Jane Miller,Sales Director
[root@node130 sed]# sed '{
> s@Developer@IT Manager@
> s@Manager@Director@
> }' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Director
103,Raj Reddy,Sysadmin
104,Anand Ram,IT Director
105,Jane Miller,Sales Director
&的作用-获取匹配到的模式
当在replacement-string中使用&时，它会被替换成匹配到的original-string或正则表达式。
[root@node130 sed]# sed 's$^[0-9][0-9][0-9]$[&]$g' employee.txt
[101],John Doe,CEO
[102],Jason Smith,IT Manager
[103],Raj Reddy,Sysadmin
[104],Anand Ram,Developer
[105],Jane Miller,Sales Manager
把每一行放进<>中
[root@node130 sed]# sed 's$^.*$<&>$' employee.txt
<101,John Doe,CEO>
<102,Jason Smith,IT Manager>
<103,Raj Reddy,Sysadmin>
<104,Anand Ram,Developer>
<105,Jane Miller,Sales Manager>

正则表达式
行的开头(^)
^匹配每一行的开头；只有^出现在正则表达式开头，他才匹配行的开头；所以^N匹配所有以N开头的行。
行的结尾($)
[root@node130 sed]# sed -n '/r$/p' employee.txt
102,Jason Smith,IT Manager
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
单个字符(.)
元字符.匹配除换行符之外的任意单个字符
.匹配单个字符
..匹配两个字符
...匹配三个字符
[root@node130 sed]# sed -n '/J... /p' employee.txt
101,John Doe,CEO
105,Jane Miller,Sales Manager
匹配0次或多次(*)
星号*匹配0个或多个其前面的字符。如：1*匹配0个或多个1.
[root@node130 sed]# sed -n '/log: *./p' log.txt
log:input.txt
log: testing resumed
log:output created
匹配一次或多次(+)
"+"匹配一次或多次它前面的字符，例如 空格+ 或"+"匹配至少一个或多个空格。
匹配0次或一次(?)
?匹配0次或一次它前面的字符。如：
[root@node130 sed]# sed -n '/log: ?/p' log.txt
log:input.txt
log:
log: testing resumed
log:
log:output created
转义字符()
如果要在正则表达式中搜寻特殊字符(如:*,.)，必须使用来转义它们。
[root@node130 sed]# sed  -n '/127.0.0.1/p' /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
字符集([0-9])
字符集匹配方括号中出现的任意一个字符；
方括号中，可以使用连接符-指定一个字符范围。如[0123456789]可以用[0-9]表示，字母可以用
[a-z],[A-Z]表示。
[root@node130 sed]# sed -n '/[2-4]/p' employee.txt
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
其他正则表达式
或操作符(|)
管道符号|用来匹配两边任意一个子表达式。子表达式1|子表达式2|子表达式3
[root@node130 sed]# sed -n '/101|102/p' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
[root@node130 sed]# sed -n '/[2-3]|105/p' employee.txt
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
105,Jane Miller,Sales Manager
精确匹配m次({m})
正则表达式后面跟上{m}标明精确匹配该正则m次。
[root@node130 sed]# cat numbers.txt
1
12
123
1234
12345
123456

[root@node130 sed]# sed -n '/^[0-9]{5}$/p' numbers.txt
sed: -e expression #1, char 13: Unmatched {
[root@node130 sed]# sed -n '/^[0-9]{5}$/p' numbers.txt
12345
[root@node130 sed]# sed -n '/^[0-9]{3}$/p' numbers.txt
123
匹配m至n次({m,n})
正则表达式后面跟上{m,n}表明精确匹配该正则至少m,最多n次。m,n不能是负数，并且要小于255.
{m,}匹配至少m;最多不限；{,n}表明最多匹配n次，最少一次。
打印有3至5个数字组成的行
[root@node130 sed]# sed -n '/^[0-9]{3,5}$/p' numbers.txt
123
1234
12345
字符边界()
[root@node130 sed]# cat words.txt
word matching using:the
word matching using:thethe
word matching using:they

[root@node130 sed]# sed -n '/the/p' words.txt
word matching using:the
[root@node130 sed]# sed -n '/the/p' words.txt
word matching using:the
word matching using:thethe
word matching using:they
在sed替换中使用正则表达式
[root@node130 sed]# cat employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 sed]# sed -n 's@..$@,Not Defined@p' employee.txt
101,John Doe,C,Not Defined
102,Jason Smith,IT Manag,Not Defined
103,Raj Reddy,Sysadm,Not Defined
104,Anand Ram,Develop,Not Defined
105,Jane Miller,Sales Manag,Not Defined
[root@node130 sed]# sed -n 's/..$/,Not Defined/p' employee.txt
101,John Doe,C,Not Defined
102,Jason Smith,IT Manag,Not Defined
103,Raj Reddy,Sysadm,Not Defined
104,Anand Ram,Develop,Not Defined
105,Jane Miller,Sales Manag,Not Defined
清楚html标签
[root@node130 sed]# cat test.html
<html><body><h1>Hello World!</h1></body></html>
[root@node130 sed]# sed 's/<[^>]*>//g' test.html
Hello World!
删除所有注释行和空行
[root@node130 sed]# sed -e 's/#.*//;/^$/d' /etc/profile
单行内执行多个sed命令
[root@node130 sed]# sed -e 'command1' -e 'command2' -e 'command3'
[root@node130 sed]# sed -n -e '/^root/p' -e '/^nobody/p' -e '/^mail/p' /etc/passwd
root:x:0:0:root:/root:/bin/bash
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
nobody:x:99:99:Nobody:/:/sbin/nologin
[root@node130 sed]# sed -n '{
> /^root/p
> /^nobody/p
> /^mail/p
> }' /etc/passwd
root:x:0:0:root:/root:/bin/bash
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
nobody:x:99:99:Nobody:/:/sbin/nologin
sed脚本命令及注释
sed注释以#开头
[root@node130 sed]# cat mycommand.sed
#交换第一列和第二列
s/([^,]*),([^,]*),(.*).*/2,1,3/g
#将整行内容放入<>中
s/^.*/<&>/
#把Developer替换为IT Manager
s/Developer/IT Manager/
#把Manager替换为Director
s/Manager/Director/
[root@node130 sed]# sed -f mycommand.sed employee.txt
<John Doe,101,CEO>
<Jason Smith,102,IT Director>
<Raj Reddy,103,Sysadmin>
<Anand Ram,104,IT Director>
<Jane Miller,105,Sales Director>
把sed当作命令解释器使用
需要在sed脚本最开始加入"#!/bin/sed -f".
[root@node130 sed]# cat myscript.sed
#!/bin/sed -f
s/([^,]*),([^,]*),(.*).*/2,1,3/g
s/^.*/<&>/
s/Developer/IT Manager/
s/Manager/Director/
[root@node130 sed]# chmod u+x myscript.sed
[root@node130 sed]# ./myscript.sed employee.txt
<John Doe,101,CEO>
<Jason Smith,102,IT Director>
<Raj Reddy,103,Sysadmin>
<Anand Ram,104,IT Director>
<Jane Miller,105,Sales Director>
[root@node130 sed]# chmod u+x testscript.sed
[root@node130 sed]# cat testscript.sed
#!/bin/sed -nf
/root/p
/nobody/p
/mail/p
[root@node130 sed]# ./testscript.sed /etc/passwd
root:x:0:0:root:/root:/bin/bash
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
nobody:x:99:99:Nobody:/:/sbin/nologin
nfsnobody:x:65534:65534:Anonymous NFS User:/var/lib/nfs:/sbin/nologin

sed直接修改输入文件 -i选项可以直接修改输入文件
[root@node130 sed]# sed -i 's/John/Johnny' employee.txt

[root@node130 ibak]# ls
employee.txt
[root@node130 ibak]# sed -ibak 's/John/Johnnyyyy/' employee.txt
[root@node130 ibak]# ls
employee.txt  employee.txtbak
[root@node130 ibak]# cat employee.txt
101,Johnnyyyy Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 ibak]# cat employee.txtbak
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
sed附加命令
追加命令(命令a)
命令a可以在指定位置的后面插入新行
语法：
sed '[address] a the-line-to-append' input-file
[root@node130 sed]# sed '2a 203,Jack Johnson,Engineer' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
203,Jack Johnson,Engineer
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 sed]# sed '$a 106,Jack Johnson,Engineer' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
106,Jack Johnson,Engineer
追加多行
[root@node130 sed]# sed '/Jason/a 203,Jack Johnson,Engineer 204,Mark Smith,Sales Engineer' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
203,Jack Johnson,Engineer
204,Mark Smith,Sales Engineer
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager

[root@node130 sed]# sed '/Jason/a
> 203,Jack Johnson,Engineer
> 204,Mark Smith,Sales Engineer' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
203,Jack Johnson,Engineer
204,Mark Smith,Sales Engineer
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
插入命令(命令i)
插入命令insert命令和追加命令类似，只是在指定位置之前插入行。
语法：
sed '[address] i the-line-to-insert' input-file
[root@node130 sed]# sed '2i 203,Jack Johnson,Engineer' employee.txt
101,John Doe,CEO
203,Jack Johnson,Engineer
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 sed]# sed '$i 108,Jack Johnson,Engineer' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
108,Jack Johnson,Engineer
105,Jane Miller,Sales Manager
修改命令(命令c)
修改命令changek可以用新行取代旧行
[root@node130 sed]# sed '2c 202,Jack,Johnson,Engineer' employee.txt
101,John Doe,CEO
202,Jack,Johnson,Engineer
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 sed]# sed '/Raj/c 203,Jack Johnson,Engineer 204,Mark Smith,Sales Engineer' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
203,Jack Johnson,Engineer
204,Mark Smith,Sales Engineer
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 sed]# sed '/Jason/{
> a
> 204,Jack Johnson,Engineer
> i
> 202,Mark Smith,Sales Engineer
> c
> 203,Joe Mason,Sysadmin
> }' employee.txt
101,John Doe,CEO
202,Mark Smith,Sales Engineer
203,Joe Mason,Sysadmin
204,Jack Johnson,Engineer
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
打印不可见的字符
[root@node130 sed]# sed -n 'l' tabfile.txt
fname First Name$
lname Last Name$
mname Middle Name$
[root@node130 sed]# cat tabfile.txt
fname First Name
lname Last Name
mname Middle Name
打印行号
[root@node130 sed]# sed '=' employee.txt
1
101,John Doe,CEO
2
102,Jason Smith,IT Manager
3
103,Raj Reddy,Sysadmin
4
104,Anand Ram,Developer
5
105,Jane Miller,Sales Manager
打印文件总行数
[root@node130 sed]# sed -n '$=' employee.txt
5
sed模拟Unix命令(cat,grep,read)
[root@node130 sed]# cat employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 sed]# sed 's/JUNK/&/p' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 sed]# sed -n 'p' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 sed]# sed 'n' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 sed]# sed 'N' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
grep
[root@node130 sed]# grep Jane employee.txt
105,Jane Miller,Sales Manager
[root@node130 sed]# sed -n 's/Jane/&/p' employee.txt
105,Jane Miller,Sales Manager
[root@node130 sed]# sed -n '/Jane/p' employee.txt
105,Jane Miller,Sales Manager
[root@node130 sed]# head -10 /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
uucp:x:10:14:uucp:/var/spool/uucp:/sbin/nologin
[root@node130 sed]# sed '11,$d' /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
uucp:x:10:14:uucp:/var/spool/uucp:/sbin/nologin
[root@node130 sed]# sed -n '1,10p' /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
uucp:x:10:14:uucp:/var/spool/uucp:/sbin/nologin
[root@node130 sed]# sed '10q' /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
uucp:x:10:14:uucp:/var/spool/uucp:/sbin/nologin
-n(--quiet,--silent):屏蔽sed的默认输出
[root@node130 sed]# sed -n 'p' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 sed]# sed --quiet 'p' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 sed]# sed --silent 'p' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
-f:调用由多个sed命令组成的sed脚本文件
[root@node130 sed]# cat testscript.sed
#!/bin/sed -f
/root/p
/nobody/p
/mail/p
[root@node130 sed]# sed -n -f testscript.sed /etc/passwd
root:x:0:0:root:/root:/bin/bash
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
nobody:x:99:99:Nobody:/:/sbin/nologin
nfsnobody:x:65534:65534:Anonymous NFS User:/var/lib/nfs:/sbin/nologin
[root@node130 sed]# sed -n --file=testscript.sed /etc/passwd
root:x:0:0:root:/root:/bin/bash
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
nobody:x:99:99:Nobody:/:/sbin/nologin
nfsnobody:x:65534:65534:Anonymous NFS User:/var/lib/nfs:/sbin/nologin
[root@node130 sed]# sed -n -e '/root/p' /etc/passwd
root:x:0:0:root:/root:/bin/bash
operator:x:11:0:operator:/root:/sbin/nologin
[root@node130 sed]# sed -n --expression '/root/p' /etc/passwd
root:x:0:0:root:/root:/bin/bash
operator:x:11:0:operator:/root:/sbin/nologin

[root@node130 ibak]# sed --in-place=bak 's/102/gooogle/' employee.txt
[root@node130 ibak]# ls
employee.txt  employee.txtbak  employee.txt.orig
[root@node130 ibak]# cat employee.txt
1000001,Johnnyyyyy Doe,CEO
gooogle,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 ibak]# sed -ibak -c 's/John/helloworld/' employee.txt
[root@node130 ibak]# ls
employee.txt  employee.txtbak
[root@node130 ibak]# cat employee.txt
101,helloworldyyyy Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 ibak]# cat employee.txtbak
101,Johnyyyy Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
打印模式空间命令(n)
命令n打印当前模式空间的内容，然后从输入文件中读取下一行。如果在命令执行过程中
遇到n,那么它会改变正常的执行流程。
[root@node130 sed]# sed n employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 sed]# sed -n n employee.txt
sed有两个内置的存储空间
模式空间：模式空间用于sed执行的正常流程中。该空间sed内置一个缓冲区，用来存放、修改从文件读取的内容。
保持空间：保持空间是另外一个缓冲区，用来存放临时数据。sed可以在保持空间和模式空间交换数据，但是
不能在保持空间上执行普通的sed命令。
每次循环读取数据过程中，模式空间的内容都会被清空，然而保持空间的内容则保持不变，不会在循环中被删除。

sed 's/^Managet.*//' employee.txt
[root@node130 sed]# sed -n '/^103/p' employee.txt
103,Raj Reddy,Sysadmin

[root@node130 sed]# sed 's//usr/local/bin//usr/
> ^C
[root@node130 sed]# sed 's//usr/local/bin//usr/
> ^C
[root@node130 sed]# sed 's//usr/local/bin//usr/
> ^C
[root@node130 sed]# sed 's//usr/local/bin//usr/bin/' path.txt
reading /usr/bin directory
[root@node130 sed]# sed 's|/usr/local/bin|/usr/bin|' path.txt
reading /usr/bin directory
[root@node130 sed]# sed 's^/usr/local/bin^/usr/bin^' path.txt
reading /usr/bin directory
[root@node130 sed]# sed 's@/usr/local/bin@/usr/bin@' path.txt
reading /usr/bin directory
[root@node130 sed]# sed 's!/usr/local/bin!/usr/bin!' path.txt
reading /usr/bin directory
使用替换标志组合
[root@node130 sed]# sed -n 's/manager/Director/igpw output_1_22.txt' employee.txt
102,Jason Smith,IT Director
105,Jane Miller,Sales Director
[root@node130 sed]# cat output_1_22.txt
102,Jason Smith,IT Director
105,Jane Miller,Sales Director

==========================================================================
awk内置变量：
FS-输入字段分隔符(待处理文件);FS只能在BEGIN区域中使用。
awk 默认的字段分隔符是空格，如果输入文件中不是一个空格作为字段分隔符，使用-F指定。
OFS-输出字段分隔符。OFS会被打印在输出行的字段之间；默认情况下，awk在输出字段中间以空格分开。
RS-记录分隔符；RS只能在BEGIN区域中使用。
ORS-输出记录分隔符；ORS只能在BEGIN区域中使用。
NR-记录序号；在block区域时代表记录的序号(Nunber of the Record)代表awk当前处理的行号;用于END区域时，代表输入文件的总记录数。
FIELNAEM-当前处理的文件名；如果awk从标准输入获取内容，FILENAME的值将会是“-”。

FS是输入字段分隔符(待处理文件)，OFS是输出字段分隔符(待输出文件)；OFS会被打印在输出行的连续的字段之间。
默认情况下awk输出字段中间以空格分开。 
[root@node130 ~]# sed -n -e '/^root/ p' -e '/^nobody/ p' /etc/passwd
root:x:0:0:root:/root:/bin/bash
nobody:x:99:99:Nobody:/:/sbin/nologin
[root@node130 ~]# sed -n
> -e '/^root/ p' /etc/passwd
> -e '/^nobody/ p' /etc/passwd
root:x:0:0:root:/root:/bin/bash
nobody:x:99:99:Nobody:/:/sbin/nologin
root:x:0:0:root:/root:/bin/bash
nobody:x:99:99:Nobody:/:/sbin/nologin
[root@node130 ~]#
[root@node130 ~]# sed -n '{/^root/ p /^nobody/ p}' /etc/passwd
sed: -e expression #1, char 12: extra characters after command
[root@node130 ~]# sed -n '{/^root/ p
> /^nobody/ p
> }'/etc/passwd
sed: -e expression #1, char 25: extra characters after command
[root@node130 ~]# sed /etc/passwd
sed: -e expression #1, char 7: extra characters after command
[root@node130 ~]# sed 'p' /etc/passwd
root:x:0:0:root:/root:/bin/bash
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
uucp:x:10:14:uucp:/var/spool/uucp:/sbin/nologin
uucp:x:10:14:uucp:/var/spool/uucp:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
gopher:x:13:30:gopher:/var/gopher:/sbin/nologin
gopher:x:13:30:gopher:/var/gopher:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
nobody:x:99:99:Nobody:/:/sbin/nologin
nobody:x:99:99:Nobody:/:/sbin/nologin
dbus:x:81:81:System message bus:/:/sbin/nologin
dbus:x:81:81:System message bus:/:/sbin/nologin
usbmuxd:x:113:113:usbmuxd user:/:/sbin/nologin
usbmuxd:x:113:113:usbmuxd user:/:/sbin/nologin
vcsa:x:69:69:virtual console memory owner:/dev:/sbin/nologin
vcsa:x:69:69:virtual console memory owner:/dev:/sbin/nologin
rtkit:x:499:497:RealtimeKit:/proc:/sbin/nologin
rtkit:x:499:497:RealtimeKit:/proc:/sbin/nologin
avahi-autoipd:x:170:170:Avahi IPv4LL Stack:/var/lib/avahi-autoipd:/sbin/nologin
avahi-autoipd:x:170:170:Avahi IPv4LL Stack:/var/lib/avahi-autoipd:/sbin/nologin
abrt:x:173:173::/etc/abrt:/sbin/nologin
abrt:x:173:173::/etc/abrt:/sbin/nologin
haldaemon:x:68:68:HAL daemon:/:/sbin/nologin
haldaemon:x:68:68:HAL daemon:/:/sbin/nologin
gdm:x:42:42::/var/lib/gdm:/sbin/nologin
gdm:x:42:42::/var/lib/gdm:/sbin/nologin
ntp:x:38:38::/etc/ntp:/sbin/nologin
ntp:x:38:38::/etc/ntp:/sbin/nologin
apache:x:48:48:Apache:/var/www:/sbin/nologin
apache:x:48:48:Apache:/var/www:/sbin/nologin
saslauth:x:498:76:"Saslauthd user":/var/empty/saslauth:/sbin/nologin
saslauth:x:498:76:"Saslauthd user":/var/empty/saslauth:/sbin/nologin
postfix:x:89:89::/var/spool/postfix:/sbin/nologin
postfix:x:89:89::/var/spool/postfix:/sbin/nologin
pulse:x:497:496:PulseAudio System Daemon:/var/run/pulse:/sbin/nologin
pulse:x:497:496:PulseAudio System Daemon:/var/run/pulse:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin
tcpdump:x:72:72::/:/sbin/nologin
tcpdump:x:72:72::/:/sbin/nologin
rhel:x:500:500:rhel:/home/rhel:/bin/bash
rhel:x:500:500:rhel:/home/rhel:/bin/bash
kaadmin:x:501:501::/home/kaadmin:/bin/bash
kaadmin:x:501:501::/home/kaadmin:/bin/bash
ilanni:x:502:502::/home/ilanni:/bin/bash
ilanni:x:502:502::/home/ilanni:/bin/bash
dovecot:x:97:97:Dovecot IMAP server:/usr/libexec/dovecot:/sbin/nologin
dovecot:x:97:97:Dovecot IMAP server:/usr/libexec/dovecot:/sbin/nologin
dovenull:x:496:493:Dovecot's unauthorized user:/usr/libexec/dovecot:/sbin/nologin
dovenull:x:496:493:Dovecot's unauthorized user:/usr/libexec/dovecot:/sbin/nologin
test:x:503:503::/home/test:/bin/bash
test:x:503:503::/home/test:/bin/bash

[root@node130 ~]# sed 'p' employee.txt
101,John Doe,CEO
101,John Doe,CEO
102,Jason Smith,IT Manager
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
105,Jane Miller,Sales Manager
[root@node130 ~]# sed -n 'p' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 ~]# sed '1~2d' employee.txt
102,Jason Smith,IT Manager
104,Anand Ram,Developer
[root@node130 ~]# sed '2~2d' employee.txt
101,John Doe,CEO
103,Raj Reddy,Sysadmin
105,Jane Miller,Sales Manager
[root@node130 ~]# sed '/Manager/ d' employee.txt
101,John Doe,CEO
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
[root@node130 ~]# sed '/Jason/,4d' employee.txt
101,John Doe,CEO
105,Jane Miller,Sales Manager
[root@node130 ~]# sed '/Raj/,$ d' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
[root@node130 ~]# sed '/Jason/,+2,d' employee.txt
sed: -e expression #1, char 11: unknown command: `,'
[root@node130 ~]# sed '/Jason/,+2 d' employee.txt
101,John Doe,CEO
105,Jane Miller,Sales Manager
[root@node130 ~]# sed '/^$/ d' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 ~]# sed '/^#/' employee.txt
sed: -e expression #1, char 4: missing command
[root@node130 ~]# sed '/^#/ d' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 ~]# ls
anaconda-ks.cfg  Documents  employee.txt  hosts   install.log         KingbaseAnalyticsDB-3.0-build1-CENTOS6-x86_64.run  Pictures  Templates  test-script.sed  Videos
Desktop          Downloads  hello         hosts2  install.log.syslog  Music                                              Public    test       tutor
[root@node130 ~]# vim hello
[root@node130 ~]# rm -rf hello
[root@node130 ~]# l
> ^C

[root@node130 awk]# awk '{print}' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 awk]# sed -n 'p' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 awk]# cat employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
awk默认使用空格作为分隔符
[root@node130 awk]# awk '{print $2}' employee.txt
Doe,CEO
Smith,IT
Reddy,Sysadmin
Ram,Developer
Miller,Sales
awk指定逗号为分隔符
[root@node130 awk]# awk -F ',' '{print $2}' employee.txt
John Doe
Jason Smith
Raj Reddy
Anand Ram
Jane Miller

[root@node130 awk]# awk -F ',' '{print $2}' employee.txt
John Doe
Jason Smith
Raj Reddy
Anand Ram
Jane Miller
[root@node130 awk]# awk -F "," '{print $2}' employee.txt
John Doe
Jason Smith
Raj Reddy
Anand Ram
Jane Miller
[root@node130 awk]# awk -F , '{print $2}' employee.txt
John Doe
Jason Smith
Raj Reddy
Anand Ram
Jane Miller

$0 代表整条记录，print默认打印整条记录
[root@node130 awk]# awk '{print}' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 awk]# awk '{print $0}' employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager

[root@node130 awk]# awk -F ',' '/Manager/{print $2,$3}' employee.txt
Jason Smith IT Manager
Jane Miller Sales Manager
[root@node130 awk]# awk -F ',' '/^102/{print "Emp id 102 is",$2}' employee.txt
Emp id 102 is Jason Smith

[root@node130 awk]# awk -F ',' '{print $2,$3}' employee.txt
John Doe CEO
Jason Smith IT Manager
Raj Reddy Sysadmin
Anand Ram Developer
Jane Miller Sales Manager
[root@node130 awk]# awk 'BEGIN {FS=","} {print $2,$3}' employee.txt
John Doe CEO
Jason Smith IT Manager
Raj Reddy Sysadmin
Anand Ram Developer
Jane Miller Sales Manager
[root@node130 awk]# awk 'BEGIN{FS=",";
> print "--------------------------------- Name Title -------------------------------"}
> {print $2," ",$3;}
> END{print "-------------------------------------"}' employee.txt

[root@node130 awk]# cat employee-multiple-fs.txt
101,John Doe;CEO%10000
102,Jason Smith;IT Manager%5000
103,Raj Reddy;Sysadmin%4500
104,Anand Ram;Developer%4500
105,Jane Miller:Sales Manager%3000
[root@node130 awk]# awk 'BEGIN{FS="[,;%]"}{print $2,$3}' employee-multiple-fs.txt
John Doe CEO
Jason Smith IT Manager
Raj Reddy Sysadmin
Anand Ram Developer
Jane Miller:Sales Manager 3000
FS是输入字段分隔符(待处理文件)，OFS是输出字段分隔符(待输出文件)；OFS会被打印在输出行的连续的字段之间。
默认情况下awk输出字段中间以空格分开。
[root@node130 awk]# cat employee.txt
101,John Doe,CEO
102,Jason Smith,IT Manager
103,Raj Reddy,Sysadmin
104,Anand Ram,Developer
105,Jane Miller,Sales Manager
[root@node130 awk]# awk 'BEGIN{FS=",";OFS=" ";ORS=" --- "}{print $1,$2,$3}' employee.txt
101
John Doe
CEO
---
102
Jason Smith
IT Manager
---
103
Raj Reddy
Sysadmin
---
104
Anand Ram
Developer
---
105
Jane Miller
Sales Manager
---
NR-(Number of the Record)记录序号；代表awk当前处理的记录的行号。
NR非常有用，在循环内部标识记录序号。用于END区域时，代表输入文件的总记录数。
[root@node130 awk]# awk 'BEGIN {FS=","}{print "Emp ld of record number",NR,"is",$1;}END{print "Total number of records:",NR}' employee.txt
Emp ld of record number 1 is 101
Emp ld of record number 2 is 102
Emp ld of record number 3 is 103
Emp ld of record number 4 is 104
Emp ld of record number 5 is 105
Total number of records: 5
FILENAME -当前出来的文件名
[root@node130 awk]# awk -F"," '{print FILENAME, $1,$2,$3}' employee.txt
employee.txt 101 John Doe CEO
employee.txt 102 Jason Smith IT Manager
employee.txt 103 Raj Reddy Sysadmin
employee.txt 104 Anand Ram Developer
employee.txt 105 Jane Miller Sales Manager
FNR-文件中的NR
[root@node130 awk]# awk 'BEGIN {FS=","}{print FIELNAME ":record number",NR,"is",$1;}END{print "Total number of records:",NR}' employee.txt employee-multiple-fs.txt
:record number 1 is 101
:record number 2 is 102
:record number 3 is 103
:record number 4 is 104
:record number 5 is 105
:record number 6 is 101
:record number 7 is 102
:record number 8 is 103
:record number 9 is 104
:record number 10 is 105
Total number of records: 10

[root@node130 awk]# cat employee-sal.txt
101,John Doe,CEO,10000
102,Jason Smith,IT Manager,5000
103,Raj Reddy,Sysadmin,4500
104,Anand Ram,Developer,4500
105,Jane Miller,Sales Manger,3000
[root@node130 awk]# cat total-company-salary.awk
BEGIN{
 FS=",";
 total=0;
}
{
 print $2"'s salary is:"$4;
 total=total+$4;
}
END{
 print "--- Total company salary=$"total;
}
[root@node130 awk]# awk -f total-company-salary.awk employee-sal.txt
John Doe's salary is:10000
Jason Smith's salary is:5000
Raj Reddy's salary is:4500
Anand Ram's salary is:4500
Jane Miller's salary is:3000
---
Total company salary=$27000
[root@node130 awk]# awk -F "," '{print $4}' employee-sal.txt
10000
5000
4500
4500
3000
[root@node130 awk]# awk -F "," '{print -$4}' employee-sal.txt
-10000
-5000
-4500
-4500
-3000
[root@node130 awk]# awk -F , '{print -$4}' employee-sal.txt
-10000
-5000
-4500
-4500
-3000
[root@node130 awk]#  awk -F, '{print ++$4}' employee-sal.txt
10001
5001
4501
4501
3001
[root@node130 awk]# awk -F, '{print --$4}' employee-sal.txt
9999
4999
4499
4499
2999
[root@node130 awk]# awk -F, '{print $4++}' employee-sal.txt
10000
5000
4500
4500
3000
[root@node130 awk]# awk -F, '{print $4--}' employee-sal.txt
10000
5000
4500
4500
3000
[root@node130 awk]# awk -F, '{$4++;print $4}' employee-sal.txt
10001
5001
4501
4501
3001
[root@node130 awk]# awk -F, '{$4--;print $4}' employee-sal.txt
9999
4999
4499
4499
2999

[root@node130 awk]# cat negative.txt
-1
-2
-3
[root@node130 awk]# awk '{print +$1}' negative.txt
-1
-2
-3
[root@node130 awk]# awk '{print -$1}' negative.txt
1
2
3
[root@node130 awk]# awk -F ':' '$NF ~//bin/bash/{n++};END {print n}' /etc/passwd
5
[root@node130 awk]# cat items.txt
101,HD Camcorder,Video,210,10
102,Refrigerator,Applian,850,2
103,MP3 Player,Audio,270,15
104,Tennis Racket,Sports,190,20
105,Laser Printer,Office,475,5
[root@node130 awk]# cat arithmetic.awk
BEGIN{
 FS=",";
 OFS=",";
 item_discont=0;
}
{
 item_discount=$4*20/100;
 print $1,$2,$3,$4-item_discount,$5-1;
}
[root@node130 awk]# awk -f arithmetic.awk items.txt
101,HD Camcorder,Video,168,9
102,Refrigerator,Applian,680,1
103,MP3 Player,Audio,216,14
104,Tennis Racket,Sports,152,19
105,Laser Printer,Office,380,4
只打印偶数行
[root@node130 awk]# awk 'NR % 2 == 0' items.txt
102,Refrigerator,Applian,850,2
104,Tennis Racket,Sports,190,20
字符串操作符-空格是连接字符串的操作符
[root@node130 awk]# cat string.awk
BEGIN{
 FS=",";
 OFS=",";
 string1="Audio";
 string2="Video";
 numberstring="100";
 string3=string1 string2;
 print "Concatenate string is:" string3;
 numberstring=numberstring+1;
 print "String to number:" numberstring;
}
[root@node130 awk]# awk -f string.awk items.txt
Concatenate string is:AudioVideo
String to number:101
赋值操作符
[root@node130 awk]# cat assignment.awk
BEGIN{
 FS=",";
 OFS=",";
 total1 = total2 = total3 = total4 = total5 = 10;
 total1 += 5;print total1;
 total2 -= 5;print total2;
 total3 *= 5;print total3;
 total4 /= 5;print total4;
 total5 %= 5;print total5;
}
[root@node130 awk]# awk -f assignment.awk
15
5
50
2
0
[root@node130 awk]# cat items.txt
101,HD Camcorder,Video,210,10
102,Refrigerator,Applian,850,2
103,MP3 Player,Audio,270,15
104,Tennis Racket,Sports,190,20
105,Laser Printer,Office,475,5
[root@node130 awk]# awk -F , 'BEGIN{total=0}{total+=$5}END{print "Total Quantity:"total}' items.txt
Total Quantity:52
[root@node130 awk]# cat items.txt
101,HD Camcorder,Video,210,10
102,Refrigerator,Applian,850,2
103,MP3 Player,Audio,270,15
104,Tennis Racket,Sports,190,20
105,Laser Printer,Office,475,5
[root@node130 awk]# awk -F ',' 'BEGIN {total=0}{total+=NF}END{print total}' items.txt
25
awk比较操作
[root@node130 awk]# awk -F',' '$5<=5' items.txt
102,Refrigerator,Applian,850,2
105,Laser Printer,Office,475,5
[root@node130 awk]# awk -F ',' '$1==103' items.txt
103,MP3 Player,Audio,270,15
[root@node130 awk]# awk -F "," '$1==103 {print $2}' items.txt
MP3 Player
[root@node130 awk]# awk -F "," '$3!="Video"' items.txt
102,Refrigerator,Applian,850,2
103,MP3 Player,Audio,270,15
104,Tennis Racket,Sports,190,20
105,Laser Printer,Office,475,5
[root@node130 awk]# awk -F "," '$3!="Video"{print $2}' items.txt
Refrigerator
MP3 Player
Tennis Racket
Laser Printer
[root@node130 awk]# awk -F"," '$4<900 && $5<=5' items.txt
102,Refrigerator,Applian,850,2
105,Laser Printer,Office,475,5
[root@node130 awk]# awk -F"," '$4<900&&$5<=5{print $2}' items.txt
Refrigerator
Laser Printer
[root@node130 awk]#  awk -F"," '$4<900 || $5<=5' items.txt
101,HD Camcorder,Video,210,10
102,Refrigerator,Applian,850,2
103,MP3 Player,Audio,270,15
104,Tennis Racket,Sports,190,20
105,Laser Printer,Office,475,5
[root@node130 awk]# awk -F"," '$4<900||$5<=5{print$2}' items.txt
HD Camcorder
Refrigerator
MP3 Player
Tennis Racket
Laser Printer
正则表达式操作符
~匹配
!~ 不匹配
==精确匹配
[root@node130 awk]# awk -F"," '$2=="Tennis"' items.txt
[root@node130 awk]# awk -F"," '$2~"Tennis"' items.txt
104,Tennis Racket,Sports,190,20
[root@node130 awk]# awk -F "," '$2 !~ "Tennis"' items.txt
101,HD Camcorder,Video,210,10
102,Refrigerator,Applian,850,2
103,MP3 Player,Audio,270,15
105,Laser Printer,Office,475,5
[root@node130 awk]# awk -F',' '{if ($5<=5)print "Only",$5,"qty of",$2"is available"}' items.txt
Only 2 qty of Refrigeratoris available
Only 5 qty of Laser Printeris available
[root@node130 awk]# awk -F',' '{if(($4>=500&&$4<=1000)&&($5<=5))print "Only",$5,"qty of",$2,"is availiable"}' items.txt
Only 2 qty of Refrigerator is availiable
[root@node130 awk]# cat if-else.awk
BEGIN{
 FS=",";
}
{
 if($5<=5)
  print "Buy More:Order",$2,"immediately!"
 else
  print "Shell More:Give discount on",$2,"immediately!"
}
[root@node130 awk]# awk -f if-else.awk items.txt
Shell More:Give discount on HD Camcorder immediately!
Buy More:Order Refrigerator immediately!
Shell More:Give discount on MP3 Player immediately!
Shell More:Give discount on Tennis Racket immediately!
Buy More:Order Laser Printer immediately!
[root@node130 awk]# awk 'ORS=NR%2?",":" "' items.txt
101,HD Camcorder,Video,210,10,102,Refrigerator,Applian,850,2
103,MP3 Player,Audio,270,15,104,Tennis Racket,Sports,190,20
105,Laser Printer,Office,475,5,[root@node130 awk]#
[root@node130 awk]# awk '{if(NR%2==0)ORS=" ";else ORS=",";print}' items.txt
101,HD Camcorder,Video,210,10,102,Refrigerator,Applian,850,2
103,MP3 Player,Audio,270,15,104,Tennis Racket,Sports,190,20
105,Laser Printer,Office,475,5,[root@node130 awk]#
[root@node130 awk]# awk 'BEGIN{while(count++<50)string=string"x";print string}'
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
[root@node130 awk]# cat while.awk
{
 i=2;total=0;
 while(i<=NF){
  total = total + $i;
  i++;
 }
 print "Item",$1,":",total,"quantities sold";
}
[root@node130 awk]# awk -f while.awk items-sold.txt
Item 101 : 47 quantities sold
Item 102 : 10 quantities sold
Item 103 : 65 quantities sold
Item 104 : 20 quantities sold
Item 105 : 42 quantities sold

[root@node130 awk]# awk 'BEGIN{
count=1;
do
print "This gets printed at least once";
while(count!=1)
}'
This gets printed at least once
[root@node130 awk]# cat dowhile.awk
{
 i=2;
 total=0;
 do{
  total=total+ $i;
  i++;
 }
 while(i<=NF)
 print "Item",$1,":",total,"quantities sold";
}
[root@node130 awk]# cat items-sold.txt
101 2 10 5 8 10 12
102 0 1 4 3 0 2
103 10 6 11 20 5 13
104 2 3 4 0 6 5
105 10 2 5 7 12 6
[root@node130 awk]# awk -f dowhile.awk items-sold.txt
Item 101 : 47 quantities sold
Item 102 : 10 quantities sold
Item 103 : 65 quantities sold
Item 104 : 20 quantities sold
Item 105 : 42 quantities sold
[root@node130 awk]# echo "1 2 3 4"|awk '{for(i=1;i<=NF;i++)total=total+$i}END{print total}'
10
[root@node130 awk]# cat for.awk
{
 total=0;
 for(i=2;i<=NF;i++)
  total=total+$i
 print "Item",$1,":",total,"quantities sold"
}
[root@node130 awk]# awk -f for.awk items-sold.txt
Item 101 : 47 quantities sold
Item 102 : 10 quantities sold
Item 103 : 65 quantities sold
Item 104 : 20 quantities sold
Item 105 : 42 quantities sold

[root@node130 awk]# cat break.awk
{
 i=2;total=0;
 while(i++<=NF)
 {
  if($i==0)
  {
   print "Item",$0,"had a month without item sold"
   break
  }
 }
}
[root@node130 awk]# awk -f break.awk items-sold.txt
Item 102 0 1 4 3 0 2  had a month without item sold
Item 104 2 3 4 0 6 5 had a month without item sold
[root@node130 awk]# awk 'BEGIN{while(1)print "forever"}'
Continue语句跳过后面剩余的循环部分，立即进入下次循环；
Continue只能用在循环当中。
[root@node130 awk]# cat continue.awk
{
 i=1;total=0;
 while(i++<=NF)
 {
  if(i==1)
   continue
   total=total+$i
 }
 print "Item",$1,":",total,"quantities sold"
}
[root@node130 awk]# awk -f continue.awk items-sold.txt
Item 101 : 47 quantities sold
Item 102 : 10 quantities sold
Item 103 : 65 quantities sold
Item 104 : 20 quantities sold
Item 105 : 42 quantities sold
exit语句-exit命令立即停止脚本的运行，并忽略脚本中其余的命令
exit命令接受一个数字参数作为awk的推出状态码，如果不提供参数，默认的状态码是0；
[root@node130 awk]# cat exit.awk
{
 i=2;total=0;
 while(i++<=NF)
  if($i==0){
   print "Item",$1,"had a month with no item sold"
   exit
  }
}
[root@node130 awk]# awk -f exit.awk items-sold.txt
Item 102 had a month with no item sold
awk 数组
[root@node130 awk]# cat array-assign.awk
BEGIN{
 item[101]="HD Camcorder";
 item[102]="Refrigerator";
 item[103]="MP3 Player"
 item[104]="Tennis Racket";
 item[105]="Laser Printer";
 item[1001]="Tennis Ball";
 item[55]="Laptop";
 item["na"]="Not Available";
 print item["101"];
 print item["102"];
 print item["103"];
 print item["104"];
 print item["105"];
 print item[1001];
        print item["na"];
}
[root@node130 awk]# awk -f array-assign.awk
HD Camcorder
Refrigerator
MP3 Player
Tennis Racket
Laser Printer
Tennis Ball
Not Available
[root@node130 awk]# cat array-refer.awk
BEGIN{
 x=item[55];
 if(55 in item)
  print "Array index 55 contains",item[55];
 item[101]="HD Camcorder";
 if(101 in item)
  print "Array index 101 contains",item[101];
 if(1010 in item)
  print "Array index 1010 contains",item[1010];
}
[root@node130 awk]# awk -f array-refer.awk
Array index 55 contains
Array index 101 contains HD Camcorder
使用循环遍历awk数组
语法：for(var in arrayname){
         actinos
   }
-:var 变量名
  in 关键字
  arrayname 数组名
  actions 一系列要执行的awk语句；通过把索引值赋给变量var，
  循环体可以把所有语句应用到数组中的所有元素上。
  例如：for(x in item) x:变量名；用来存放数组索引
  [root@node130 awk]# cat array-for-loop.awk
BEGIN{
 item[101]="HD Camcorder";
 item[102]="Refrigerator";
 item[103]="MP3 Player";
 item[104]="Tennis Racket";
 item[105]="Laser Printer";
 item[1001]="Tennis Ball";
 item[55]="Laptop";
 item["no"]="Not Available";

 for(x in item)
  print item[x]
}

[root@node130 awk]# awk -f array-for-loop.awk
Not Available
Laptop
HD Camcorder
Refrigerator
MP3 Player
Tennis Racket
Laser Printer
Tennis Ball
[root@node130 awk]# awk -f array-for-loop.awk
Index no contains Not Available
Index 55 contains Laptop
Index 101 contains HD Camcorder
Index 102 contains Refrigerator
Index 103 contains MP3 Player
Index 104 contains Tennis Racket
Index 105 contains Laser Printer
Index 1001 contains Tennis Ball
删除数组元素
语法：delete arraynaem[index];
删除数组内所有元素
for(var in array)
    delete array[var]
gawk中语法：Delete array
[root@node130 awk]# cat array-delete.awk
BEGIN{
 item[101]="HD Camcorder";
        item[102]="Refrigerator";
        item[103]="MP3 Player";
        item[104]="Tennis Racket";
        item[105]="Laser Printer";
        item[1001]="Tennis Ball";
        item[55]="Laptop";
        item["no"]="Not Available";
 
 delete item[102]
 item[103]=""
 delete item[104]
 delete item[1001]
 delete item["na"]
 
 for(x in item)
  print "Index",x,"contains",item[x]
}
[root@node130 awk]# awk -f array-delete.awk
Index no contains Not Available
Index 55 contains Laptop
Index 101 contains HD Camcorder
Index 103 contains
Index 105 contains Laser Printer