1.处理一下文件内容，将域名提取并根据域名进行数据排序处理
[root@salt-master test]# cat oldboy.txt
http://www.etiantian.org/index.html
http://www.etiantian.org/1.html
http://post.etiantian.org/index.html
http://www.etiantian.org/3.html
http://mp3.etiantian.org/index.html
http://post.etiantian.org/2.html
解答：
[root@salt-master test]# awk -F / '{print $3}' oldboy.txt|sort -r|uniq -c
3 www.etiantian.org
2 post.etiantian.org
1 mp3.etiantian.org
[root@salt-master test]# cut -d / -f3 oldboy.txt|sort -r|uniq -c
3 www.etiantian.org
2 post.etiantian.org
1 mp3.etiantian.org
2.统计高并发web服务器不同的网络连接状态对应的数量
Proto Recv-Q Send-Q Local Address Foreign Address State tcp 0 0 0.0.0.0:42081 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN tcp 0 1 172.30.0.54:47570 74.125.204.102:443 SYN_SENT tcp 0 1 172.30.0.54:47564 74.125.204.102:443 SYN_SENT tcp 0 1 172.30.0.54:47566 74.125.204.102:443 SYN_SENT tcp 0 0 172.30.0.54:32840 165.254.134.121:80 ESTABLISHED tcp 0 0 192.168.1.125:49202 192.168.1.142:139 ESTABLISHED tcp 0 1 172.30.0.54:47560 74.125.204.102:443 SYN_SENT tcp 0 1 172.30.0.54:47562 74.125.204.102:443 SYN_SENT tcp 0 1 172.30.0.54:47568 74.125.204.102:443 SYN_SENT tcp6 0 0 :::56937 :::* LISTEN tcp6 0 0 :::3306 :::* LISTEN tcp6 0 0 :::111 :::* LISTEN tcp6 0 0 ::1:631 :::* LISTEN
解答：
[root@salt-master test]# awk '/^tcp/ {print $NF}' netstat.log|sort|uniq -c|sort -rn|head
7 LISTEN
6 SYN_SENT
2 ESTABLISHED
[root@salt-master test]# awk '/^tcp/ {S[$NF]++}END{for(k in S) print S[k],k}' netstat.log|sort -rn|head
7 LISTEN
6 SYN_SENT
2 ESTABLISHED
3.分析图片服务日志，把日志（每个图片访问次数*图片大小的总和）排行，取top10，也就是计算每个url的总访问大小
说明：本题生产环境应用：这个功能可以用于IDC网站流量带宽很高，然后通过分析服务器日志哪些元素占用流量过大，进而进行优化或裁剪该图片，压缩js等措施。
输出格式：【访问次数*单个文件次数】 【访问次数】 文件名
[root@salt-master test]# cat access_log
59.33.26.105 - - [08/Dec/2010:15:43:56 +0800] "GET /static/images/photos/2.jpg HTTP/1.1" 200 11299 "http://oldboy.blog.51cto.com/static/web/column/17/index.shtml?courseId=43" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)"
59.33.26.105 - - [08/Dec/2010:15:43:56 +0800] "GET /static/images/photos/2.jpg HTTP/1.1" 200 11299 "http://oldboy.blog.51cto.com/static/web/column/17/index.shtml?courseId=43" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)"
59.33.26.105 - - [08/Dec/2010:15:44:02 +0800] "GET /static/flex/vedioLoading.swf HTTP/1.1" 200 3583 "http://oldboy.blog.51cto.com/static/flex/AdobeVideoPlayer.swf?width=590&height=328&url=/`DYNAMIC`/2" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)"
124.115.4.18 - - [08/Dec/2010:15:44:15 +0800] "GET /?= HTTP/1.1" 200 46232 "-" "-"
124.115.4.18 - - [08/Dec/2010:15:44:25 +0800] "GET /static/js/web_js.js HTTP/1.1" 200 4460 "-" "-"
124.115.4.18 - - [08/Dec/2010:15:44:25 +0800] "GET /static/js/jquery.lazyload.js HTTP/1.1" 200 1627 "-" "-"
解决：
[root@salt-master test]# awk '{print $7" " $10}' access_log|sort|uniq -c|awk '{print $1*$3,$1,$2}'|sort -rn|head
46232 1 /?=
22598 2 /static/images/photos/2.jpg
4460 1 /static/js/web_js.js
3583 1 /static/flex/vedioLoading.swf
1627 1 /static/js/jquery.lazyload.js
0 5
[root@salt-master test]# awk '{array_num[$7]++;array_size[$7]+=$10}END{for(x in array_num){print array_size[x],array_num[x],x}}' access_log |sort -rn -k1|head -10
46232 1 /?=
22598 2 /static/images/photos/2.jpg
4460 1 /static/js/web_js.js
3583 1 /static/flex/vedioLoading.swf
1627 1 /static/js/jquery.lazyload.js
0 5
命令解析：
读access.log日志第一行为例
$7===/static/images/photos/2.jpg
$10===11299
第一列：
array_num["/static/images/photos/2.jpg"]=1
第二列：
array_size[$7]=array_size[$7]+$10
array_size["/static/images/photos/2.jpg"]=11299