zabbix监控Nginx访问日志中的状态码

一、介绍：

由于生产环境中Nginx访问日志很多，我们需要随时监控Nginx服务器返回的状态码，方便我们能及时定位相关问题。

以下是按照分钟对数据进行抓取

二、Zabbix_Agentd创建监控脚本

1) 创建脚本之前核对Nginx的日志格式；

我这里Nginx日志格式如下，使用 "" 分割日志参数。

log_format  main  ' $http_x_forwarded_for" "$remote_user" "[$time_local]" "$request"'
                  ' "$status" "$body_bytes_sent" "$http_referer"'
                  ' "$http_user_agent" "$remote_addr" "$gzip_ratio"'
                  ' "$upstream_addr" "$request_time" "$upstream_response_time" "$http_host"';
 access_log  logs/access.log  main;

输出日志格式如下：

root@mycentos scripts]# cat /var/log/nginx/access.log | tail -n10
"-" "-" "[18/Jul/2020:11:45:41 +0800]" "GET / HTTP/1.1" "200" "540" "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36 OPR/69.0.3686.57" "43.243.94.205" "-" "-" "0.000" "-" "159.138.9.157"
"-" "-" "[18/Jul/2020:11:45:41 +0800]" "GET / HTTP/1.1" "200" "540" "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36 OPR/69.0.3686.57" "43.243.94.205" "-" "-" "0.000" "-" "159.138.9.157"
"-" "-" "[18/Jul/2020:11:45:41 +0800]" "GET / HTTP/1.1" "200" "540" "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36 OPR/69.0.3686.57" "43.243.94.205" "-" "-" "0.000" "-" "159.138.9.157"
"-" "-" "[18/Jul/2020:11:45:41 +0800]" "GET / HTTP/1.1" "200" "540" "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36 OPR/69.0.3686.57" "43.243.94.205" "-" "-" "0.000" "-" "159.138.9.157"
"-" "-" "[18/Jul/2020:11:45:41 +0800]" "GET / HTTP/1.1" "200" "540" "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36 OPR/69.0.3686.57" "43.243.94.205" "-" "-" "0.000" "-" "159.138.9.157"
"-" "-" "[18/Jul/2020:11:45:41 +0800]" "GET / HTTP/1.1" "200" "540" "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36 OPR/69.0.3686.57" "43.243.94.205" "-" "-" "0.000" "-" "159.138.9.157"
"-" "-" "[18/Jul/2020:11:45:42 +0800]" "GET / HTTP/1.1" "200" "540" "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36 OPR/69.0.3686.57" "43.243.94.205" "-" "-" "0.000" "-" "159.138.9.157"
"-" "-" "[18/Jul/2020:11:45:42 +0800]" "GET / HTTP/1.1" "200" "540" "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36 OPR/69.0.3686.57" "43.243.94.205" "-" "-" "0.000" "-" "159.138.9.157"
"-" "-" "[18/Jul/2020:11:45:42 +0800]" "GET / HTTP/1.1" "200" "540" "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36 OPR/69.0.3686.57" "43.243.94.205" "-" "-" "0.000" "-" "159.138.9.157"
"-" "-" "[18/Jul/2020:11:45:42 +0800]" "GET / HTTP/1.1" "200" "540" "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36 OPR/69.0.3686.57" "43.243.94.205" "-" "-" "0.000" "-" "159.138.9.157"

2) 创建日志监控脚本：

vim /usr/local/zabbix/scripts/ngx_logs.sh

#!/usr/bin/env bash
# -----------------------------------
# Script name   : nginx logs status code monitor
# Author        : xiaoyige
# Contact me    : xiaoyige@qq.com
# Last Modified : Jun, 18th, 2020
# -----------------------------------

[ ! -d /tmp/nginx ] && mkdir /tmp/nginx
LOG_PATH=/var/log/nginx/access.log                      #Nginx日志路径,根据自己Nginx日志路径进行修改
LOG_TEMP=/tmp/nginx/nginx_last_min.log                  #Nginx上一分钟文件
LOG_STAT=/tmp/nginx/nginx_stat.txt                      #Nginx状态码文件
LAST_MIN=`date -d "1 minute ago" +%Y:%H:%M`             #获取上一分钟值

tail -1000 ${LOG_PATH} | grep "${LAST_MIN}" > ${LOG_TEMP}  #tail 1000行数据然后进行过滤上一分钟，如果请求量较大则加大行数，过滤后将数据重定向到上一分钟文件中
cat ${LOG_TEMP} | awk -F '" "' '{print $5}' | sort | uniq -c | sort -rn > ${LOG_STAT}   #过滤上一分钟文件的状态码并对状态码进行排序去重然后显示状态码次数
# 备注 awk -F '" "' '{print $5}' 需要根据自己日志输出情况具体分析，
#200 Code
#过滤临时文件中状态码等于200的值然后打印其次数后赋值给c_200，然后重定向到/tmp/nginx/nginx_200.txt,如果其值为空，则赋值为0后重定向到/tmp/nginx/nginx_200.txt
c_200=`cat ${LOG_STAT} | awk '$2==200{print $1}'`;[ -z ${c_200} ] && c_200=0;echo ${c_200} > /tmp/nginx/nginx_200.txt
c_202=`cat ${LOG_STAT} | awk '$2==202{print $1}'`;[ -z ${c_202} ] && c_202=0;echo ${c_202} > /tmp/nginx/nginx_202.txt

#300 Code
c_301=`cat ${LOG_STAT} | awk '$2==301{print $1}'`;[ -z ${c_301} ] && c_301=0;echo ${c_301} > /tmp/nginx/nginx_301.txt
c_302=`cat ${LOG_STAT} | awk '$2==302{print $1}'`;[ -z ${c_302} ] && c_302=0;echo ${c_302} > /tmp/nginx/nginx_302.txt
c_304=`cat ${LOG_STAT} | awk '$2==304{print $1}'`;[ -z ${c_304} ] && c_304=0;echo ${c_304} > /tmp/nginx/nginx_304.txt

#400 Code
c_400=`cat ${LOG_STAT} | awk '$2==400{print $1}'`;[ -z ${c_400} ] && c_400=0;echo ${c_400} > /tmp/nginx/nginx_400.txt
c_403=`cat ${LOG_STAT} | awk '$2==403{print $1}'`;[ -z ${c_403} ] && c_403=0;echo ${c_403} > /tmp/nginx/nginx_403.txt
c_404=`cat ${LOG_STAT} | awk '$2==404{print $1}'`;[ -z ${c_404} ] && c_404=0;echo ${c_404} > /tmp/nginx/nginx_404.txt
c_405=`cat ${LOG_STAT} | awk '$2==405{print $1}'`;[ -z ${c_405} ] && c_405=0;echo ${c_405} > /tmp/nginx/nginx_405.txt

#500 Code
c_502=`cat ${LOG_STAT} | awk '$2==502{print $1}'`;[ -z ${c_502} ] && c_502=0;echo ${c_502} > /tmp/nginx/nginx_502.txt
c_503=`cat ${LOG_STAT} | awk '$2==503{print $1}'`;[ -z ${c_503} ] && c_503=0;echo ${c_503} > /tmp/nginx/nginx_503.txt
c_504=`cat ${LOG_STAT} | awk '$2==504{print $1}'`;[ -z ${c_504} ] && c_504=0;echo ${c_504} > /tmp/nginx/nginx_504.txt

#以下来定义函数方便 UserParameter 调用
function c_200 {
        cat /tmp/nginx/nginx_200.txt
}

function c_202 {
        cat /tmp/nginx/nginx_202.txt
}

function c_301 {
        cat /tmp/nginx/nginx_301.txt
}

function c_302 {
        cat /tmp/nginx/nginx_302.txt
}

function c_304 {
        cat /tmp/nginx/nginx_304.txt
}                      

function c_400 {
        cat /tmp/nginx/nginx_400.txt
}

function c_403 {
        cat /tmp/nginx/nginx_403.txt
}

function c_404 {
        cat /tmp/nginx/nginx_404.txt
}

function c_405 {
        cat /tmp/nginx/nginx_405.txt
}

function c_502 {
        cat /tmp/nginx/nginx_502.txt
}

function c_503 {
        cat /tmp/nginx/nginx_503.txt
}

function c_504 {
        cat /tmp/nginx/nginx_504.txt
}

$1

3) 修改权限属性

如果你的zabbix使用zabbix用户进启动的按照下面进行修改权限
chown -Rf zabbix.zabbix /usr/local/zabbix/scripts/ngx_logs.sh
chmod u+x /usr/local/zabbix/scripts/ngx_logs.sh
如果zabbix是使用root用户创建的
chmod +x /usr/local/zabbix/scripts/ngx_logs.sh

创建Nginx日志键值

vim /etc/zabbix/zabbix_agentd.d/userparameter_ngx_logs.conf

UserParameter=ngx.logs[*],/usr/local/zabbix/scripts/ngx_logs.sh $1

重启zabbix-agent

systemctl resart zabbix-agent

4)测试数据获取

1.本地测试数据获取

/usr/local/zabbix/scripts/ngx_logs.sh c_200
28
/usr/local/zabbix/scripts/ngx_logs.sh c_202
0
/usr/local/zabbix/scripts/ngx_logs.sh c_301
0
/usr/local/zabbix/scripts/ngx_logs.sh c_302
2
/usr/local/zabbix/scripts/ngx_logs.sh c_304
14
/usr/local/zabbix/scripts/ngx_logs.sh c_400
0
/usr/local/zabbix/scripts/ngx_logs.sh c_403
1
/usr/local/zabbix/scripts/ngx_logs.sh c_404
0
/usr/local/zabbix/scripts/ngx_logs.sh c_405
0
/usr/local/zabbix/scripts/ngx_logs.sh c_502
0
/usr/local/zabbix/scripts/ngx_logs.sh c_503
0
/usr/local/zabbix/scripts/ngx_logs.sh c_504
0

三、Zabbix_Web创建模板及监控项

1）创建模板

主页点击配置 ------> 模板------>创建模板

 2）创建应用集

 3）创建监控项

进入模板后------->监控项-------->创建监控项

 创建好后如下：

 4）创建触发器

对进程监控添加触发器，触发器——》创建触发器
填入触发器名称，此名称是告警出的信息——》选择严重性——》添加表达式——》我这里是使用了last函数最新的值如果大于15则触发告警，恢复表达式为last函数最新的至小于15则恢复告警。

 创建好后如下：

 5）创建图形

把Nginx日志监控项放在图形中

 6）主机嵌套模板

配置——>主机——>进入需要监控Nginx性能的主机——>模板——>添加模板——>选中我们创建的模板

 7）查看数据

监测——》最新数据——》选中节点——》选中应用集

 通过图形查看数据：