Goaccess解析nginx日志备忘

参考

http://nginx.org/en/docs/http/ngx_http_log_module.html?&_ga=1.92028562.949762386.1481787781#log_format
https://www.goaccess.io/man#custom-log

ssh root@server 'zcat logs/access*.gz' | goaccess --log-format='%h [%d:%t %^] "%r" %s %b "%R" "%u" %T' --date-format=%d/%b/%Y --time-format=%T -o dist/report.html

展开

goaccess默认的日志格式与nginx的默认格式一致

%h %^[%d:%t %^] "%r" %s %b "%R" "%u"

log_format combined '$remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent"';

自定义日志格式

现在增加一个响应时间, 删除$remote_user, 并把$body_bytes_sent修改为$bytes_sent,

修改nginx.conf

log_format myfmt '$remote_addr [$time_local] '
                    '"$request" $status $bytes_sent '
                    '"$http_referer" "$http_user_agent" $request_time';
access_log logs/access.log myfmt;

修改goaccess 配置文件

# vim /usr/local/etc/goaccess.conf 
time-format %H:%M:%S
date-format %d/%b/%Y
log-format %h [%d:%t %^] "%r" %s %b "%R" "%u" %T

执行示例

cat logs/access.log | goaccess -o ~/static/report.html

goaccess -f logs/access.log -o ~/static/report.html --real-time-html

变量说明

1.$remote_addr 与$http_x_forwarded_for 用以记录客户端的ip地址；
 2.$remote_user ：用来记录客户端用户名称；
 3.$time_local ： 用来记录访问时间与时区；
 4.$request ： 用来记录请求的url与http协议；
 5.$status ： 用来记录请求状态；成功是200，
 6.$body_bytes_sent ：记录发送给客户端文件主体内容大小；
 7.$http_referer ：用来记录从那个页面链接访问过来的；
 8.$http_user_agent ：记录客户端浏览器的相关信息；

nginx日志相关

Syntax: log_format name [escape=default|json] string ...;
Default:
log_format combined "...";
Context: http
Specifies log format.

The escape parameter (1.11.8) allows setting json or default characters escaping in variables, by default, default escaping is used.

The log format can contain common variables, and variables that exist only at the time of a log write:

$bytes_sent
the number of bytes sent to a client
$connection
connection serial number
$connection_requests
the current number of requests made through a connection (1.1.18)
$msec
time in seconds with a milliseconds resolution at the time of the log write
$pipe
“p” if request was pipelined, “.” otherwise
$request_length
request length (including request line, header, and request body)
$request_time
request processing time in seconds with a milliseconds resolution; time elapsed between the first bytes were read from the client and the log write after the last bytes were sent to the client
$status
response status
$time_iso8601
local time in the ISO 8601 standard format
$time_local
local time in the Common Log Format
In the modern nginx versions variables $status (1.3.2, 1.2.2), $bytes_sent (1.3.8, 1.2.5), $connection (1.3.8, 1.2.5), $connection_requests (1.3.8, 1.2.5), $msec (1.3.9, 1.2.6), $request_time (1.3.9, 1.2.6), $pipe (1.3.12, 1.2.7), $request_length (1.3.12, 1.2.7), $time_iso8601 (1.3.12, 1.2.7), and $time_local (1.3.12, 1.2.7) are also available as common variables.
Header lines sent to a client have the prefix “sent_http_”, for example, $sent_http_content_range.

The configuration always includes the predefined “combined” format:

log_format combined '$remote_addr - $remote_user [$time_local] '
'"$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent"';