准备:
1、Logstash自定义grok正则匹配规则配置文件
logstash/vendor/bundle/jruby/2.5.0/gems/logstash-patterns-core-4.1.2/patterns/grok-patterns
2、在线调试grok规则匹配网址
https://grokdebug.herokuapp.com
1、Nginx自定义grok规则
Nginx %{NGX:http_x_forwarded_for} | %{NGX:time_local} | %{NGX:status} | %{NGX:body_bytes_sent} | %{NGX:gzip_ratio} | %{NGX:request_method} | %{NGX:scheme} | %{NGX:server_protocol} | %{NGX:server_name} | %{NGX:server_port} | %{NGX:request_uri} | %{NGX:request_time} | %{NGX:content_length} | %{NGX:http_referer} | %{NGX:http_user_agent} |(s*)%{NGX:remote_addr} | %{NGX:remote_port} |(s*)%{NGX:remote_user} | %{NGX:http_cookie} | %{NGX:hostname} | %{NGX:upstream_status} | %{NGX:upstream_addr} | %{NGX:upstream_http_host} | %{NGX1:upstream_response_time}
2、Centos系统日志自定义grok规则
SYSTEMLOG #s+Time: %{GREEDYDATA:time}s+#s+User@Host:s+%{WORD:user1}[%{WORD:user2}]s+@s+[(?:%{IP:clientip})?]s+Id:s+%{NUMBER:id:number}s+#s+Query_time:s+%{NUMBER:query_time:number}s+Lock_time:s+%{NUMBER:lock_time:number}s+Rows_sent:s+%{NUMBER:rows_sent:number}s+Rows_examined:s+%{NUMBER:rows_examined:number}s+(uses+%{GREEDYDATA:usedb};s+)*SETs+timestamp=%{NUMBER:timestamp:time};s+(?<query>(?<action>w+)s+.*;)