zoukankan html css js c++ java

测试必会之 Linux 三剑客之 grep

grep (global search regular expression(RE) and print out the line,全面搜索正则表达式并把行打印出来)是一种强大的文本搜索工具，它能使用正则表达式搜索文本，并把匹配的行打印出来

可用于数据查找定位

先列举出测试工作常用的grep命令和意义：

grep pattern file
grep -i pattern file 忽略大小写
grep -v pattern file 不显示匹配行
grep -o pattern file 只把每个匹配的内容独立的行显示
grep -E pattern file 使用拓展正则表达式,相当于egrep
#注意：grep 'a[0-9]{10}' 等同于 grep -E 'a[0-9]{10}' 
grep  -B, --before-context=NUM  打印以文本起始的NUM 行
grep  -A, --after-context=NUM   打印以文本结尾的NUM 行
grep  -C, --context=NUM         打印输出文本NUM 行
grep pattern -r dir/ 递归搜索
grep -m1 匹配匹配中的第一个
grep -n 顺便输出行号
grep -P 可以让grep使用perl的正则表达式语法

BRE基本正则表达式：

^开头 $结尾
[a-z] [0-9]区间，如果开头带有^表示不能匹配区间内的元素
*0个或多个
.表示任意字符

ERE扩展正则表达式

ERE扩展表达式是基本正则表达式（BRE）基础上的扩展

？非贪婪匹配
+ 一个或多个
() 分组
{} 范围约束
| 匹配多个表达式的任意一个

话不多说，先举例

假设有如下文件，名称为nginx.log

40.77.192.252 - - [05/Dec/2018:00:08:51 +0000] "GET /uploads/photo/2017/ed39a6f0-8111-48bc-88e0-604939b1dca1.png!large HTTP/1.1" 200 10065 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534+ (KHTML, like Gecko) BingPreview/1.0b" 0.000 0.000 .
220.181.108.187 - - [05/Dec/2018:00:09:04 +0000] "GET /topics/15275?locale=en HTTP/1.1" 200 28527 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" 0.090 0.090 .
116.25.42.125 - - [05/Dec/2018:00:09:17 +0000] "GET /cable HTTP/1.1" 101 1473 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3622.0 Safari/537.36" 108.100 108.100 .
123.127.112.18 - - [05/Dec/2018:00:09:18 +0000] "GET /cable HTTP/1.1" 101 1017 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36" 70.577 70.577 .
139.180.132.174 - - [05/Dec/2018:00:09:20 +0000] "GET /bbs.zip HTTP/1.1" 404 1264 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 0.011 0.011 .
139.180.132.174 - - [05/Dec/2018:00:09:12 +0000] "GET /__zep__/js.zip HTTP/1.1" 500 2183 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 0.018 0.018 .
141.8.142.131 - - [05/Dec/2018:00:09:12 +0000] "GET /topics/14442 HTTP/1.1" 200 21980 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)" 0.195 0.195 .
220.181.108.181 - - [05/Dec/2018:00:09:13 +0000] "GET /syyair/following?locale=zh-CN HTTP/1.1" 200 13355 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" 0.035 0.035 .
139.180.132.174 - - [05/Dec/2018:00:09:13 +0000] "GET /admin.rar HTTP/1.1" 401 50 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 0.006 0.006 .
40.77.167.1 - - [05/Dec/2018:00:09:17 +0000] "GET /topics/2547/replies/17243/reply_suggest HTTP/1.1" 404 3530 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" 0.014 0.014 .
139.180.132.174 - - [05/Dec/2018:00:09:17 +0000] "GET /admin.tar.gz HTTP/1.1" 404 1264 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 0.052 0.052 .

找出500错误时候的上下文

chenshifengdeMacBook-Pro:~ chenshifeng$ grep -A 2 ' 500 ' nginx.log    #显示500错误及其后2行
139.180.132.174 - - [05/Dec/2018:00:09:12 +0000] "GET /__zep__/js.zip HTTP/1.1" 500 2183 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 0.018 0.018 .
141.8.142.131 - - [05/Dec/2018:00:09:12 +0000] "GET /topics/14442 HTTP/1.1" 200 21980 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)" 0.195 0.195 .
220.181.108.181 - - [05/Dec/2018:00:09:13 +0000] "GET /syyair/following?locale=zh-CN HTTP/1.1" 200 13355 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" 0.035 0.035 .
chenshifengdeMacBook-Pro:~ chenshifeng$ grep -B 2 ' 500 ' nginx.log    #显示500错误及其前2行
123.127.112.18 - - [05/Dec/2018:00:09:18 +0000] "GET /cable HTTP/1.1" 101 1017 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36" 70.577 70.577 .
139.180.132.174 - - [05/Dec/2018:00:09:20 +0000] "GET /bbs.zip HTTP/1.1" 404 1264 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 0.011 0.011 .
139.180.132.174 - - [05/Dec/2018:00:09:12 +0000] "GET /__zep__/js.zip HTTP/1.1" 500 2183 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 0.018 0.018 .
chenshifengdeMacBook-Pro:~ chenshifeng$ grep -C 2 ' 500 ' nginx.log    #显示500错误及其前后2行
123.127.112.18 - - [05/Dec/2018:00:09:18 +0000] "GET /cable HTTP/1.1" 101 1017 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36" 70.577 70.577 .
139.180.132.174 - - [05/Dec/2018:00:09:20 +0000] "GET /bbs.zip HTTP/1.1" 404 1264 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 0.011 0.011 .
139.180.132.174 - - [05/Dec/2018:00:09:12 +0000] "GET /__zep__/js.zip HTTP/1.1" 500 2183 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 0.018 0.018 .
141.8.142.131 - - [05/Dec/2018:00:09:12 +0000] "GET /topics/14442 HTTP/1.1" 200 21980 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)" 0.195 0.195 .
220.181.108.181 - - [05/Dec/2018:00:09:13 +0000] "GET /syyair/following?locale=zh-CN HTTP/1.1" 200 13355 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" 0.035 0.035 .

1、先访问博客主页，利用 grep href 过滤出所有包含 url 的内容。命令：

chenshifengdeMacBook-Pro:~ chenshifeng$ curl -s https://www.cnblogs.com/feng0815/ | grep -o "https:[^"]*"
https://www.cnblogs.com/feng0815/rss
https://www.cnblogs.com/feng0815/rsd.xml
https://www.cnblogs.com/feng0815/wlwmanifest.xml
https://common.cnblogs.com/scripts/jquery-2.2.0.min.js
https://mathjax.cnblogs.com/2_7_5/MathJax.js?config=TeX-AMS-MML_HTMLorMML&amp;v=20200504
https://www.cnblogs.com/
https://news.cnblogs.com/
https://q.cnblogs.com/
https://brands.cnblogs.com/
https://ing.cnblogs.com/
https://edu.cnblogs.com/
https://zzk.cnblogs.com/s
https://passport.cnblogs.com/GetBlogApplyStatus.aspx
https://msg.cnblogs.com/
https://home.cnblogs.com/
https://passport.cnblogs.com/GetBlogApplyStatus.aspx
https://home.cnblogs.com/
https://account.cnblogs.com/settings/account
https://account.cnblogs.com/signup/
......

再用grep命令匹配"200 OK"作为判断条件，筛选出成功的url并打印，然后将失败的 url 加上 ERR 标记也一起打印出来。

chenshifengdeMacBook-Pro:~ chenshifeng$ curl -s https://www.cnblogs.com/feng0815/ | grep -o "https:[^"]*"| while read line;do curl -s -I $line | grep ' 200 ' && echo $line || echo ERR $line;done
ERR https://www.cnblogs.com/feng0815/rss
HTTP/1.1 200 OK
https://www.cnblogs.com/feng0815/rsd.xml
HTTP/1.1 200 OK
https://www.cnblogs.com/feng0815/wlwmanifest.xml
HTTP/1.1 200 OK
https://common.cnblogs.com/scripts/jquery-2.2.0.min.js
HTTP/1.1 200 OK
https://mathjax.cnblogs.com/2_7_5/MathJax.js?config=TeX-AMS-MML_HTMLorMML&amp;v=20200504
HTTP/1.1 200 OK
https://www.cnblogs.com/
HTTP/1.1 200 OK
https://news.cnblogs.com/
HTTP/1.1 200 OK
https://q.cnblogs.com/
HTTP/1.1 200 OK
https://brands.cnblogs.com/
HTTP/1.1 200 OK
https://ing.cnblogs.com/
HTTP/1.1 200 OK
https://edu.cnblogs.com/
HTTP/1.1 200 OK
https://zzk.cnblogs.com/s
ERR https://passport.cnblogs.com/GetBlogApplyStatus.aspx
ERR https://msg.cnblogs.com/
HTTP/1.1 200 OK
https://home.cnblogs.com/
ERR https://passport.cnblogs.com/GetBlogApplyStatus.aspx
......

查看全文

相关阅读:
数学+高精度 ZOJ 2313 Chinese Girls' Amusement
最短路(Bellman_Ford) POJ 1860 Currency Exchange
贪心 Gym 100502E Opening Ceremony
概率 Gym 100502D Dice Game
判断 Gym 100502K Train Passengers
BFS POJ 3278 Catch That Cow
DFS POJ 2362 Square
DFS ZOJ 1002/HDOJ 1045 Fire Net
组合数学(全排列)+DFS CSU 1563 Lexicography
stack UVA 442 Matrix Chain Multiplication

原文地址：https://www.cnblogs.com/feng0815/p/13585716.html