zoukankan      html  css  js  c++  java
  • logstash使用grok过滤数据

    有一段线上日志:

    2020-07-14 11:37:04.556 INFO  [com.lyf.action.PlayAction:124] - [ 播放日志 add ] userid: 0 vid: 8079245, vtime: -1
    ┏━━━━━ Debug [native.update d_stcs_month_page set vcount = ifnul...] ━━━
    ┣ SQL:  update d_stcs_month_page set vcount = ifnull(vcount,0) + 1 where deptid = ? and vyear = ? and vmonth = ?
    ┣ 参数:         [0, 2020, 7]
    ┣ 位置:         com.chaoxing.video.mobile.front.service.stcs.PageStcsThreadTask.run(PageStcsThreadTask.java:76)
    ┣ 时间:         1ms
    ┣ 更新:         [1]
    ┗━━━━━ Debug [native.update d_stcs_month_page set vcount = ifnul...] ━━━
    

    想把log打印的内容部分通过grok读取出来(前提:使用了多行合并),也就是[ 播放日志 add ] userid: 0 vid: 8079245, vtime: -1这一段,下面是过滤规则:

    filter {
      if [type] == "debug" {
        grok {
          match => ["message", "%{TIMESTAMP_ISO8601:logdate} %{LOGLEVEL:level} (?<class>.*) - (?<log>[^┏]*)(?<sql>.*)"]
        }
        date {
          match => ["logdate", "yyyy-MM-dd HH:mm:ss.SSS"]
        }
      }
    }
    

    grokdebug分析结果如下:

    {
      "logdate": [
        [
          "2020-07-14 11:37:04.556"
        ]
      ],
      "YEAR": [
        [
          "2020"
        ]
      ],
      "MONTHNUM": [
        [
          "07"
        ]
      ],
      "MONTHDAY": [
        [
          "14"
        ]
      ],
      "HOUR": [
        [
          "11",
          null
        ]
      ],
      "MINUTE": [
        [
          "37",
          null
        ]
      ],
      "SECOND": [
        [
          "04.556"
        ]
      ],
      "ISO8601_TIMEZONE": [
        [
          null
        ]
      ],
      "level": [
        [
          "INFO"
        ]
      ],
      "class": [
        [
          " [com.lyf.action.PlayAction:124]"
        ]
      ],
      "log": [
        [
          "[ 播放日志 add ] userid: 0 vid: 8079245, vtime: -1
    "
        ]
      ],
      "sql": [
        [
          "┏━━━━━ Debug [native.update d_stcs_month_page set vcount = ifnul...] ━━━"
        ]
      ]
    }
    
  • 相关阅读:
    css样式学习笔记
    Css教程玉女心经版本
    weblogic高级进阶之ssl配置证书
    weblogic高级进阶之查看日志
    weblogic之高级进阶JMS的应用
    【WebLogic使用】3.WebLogic配置jndi数据源
    shiro的helloworld
    尚硅谷spring 事物管理
    尚硅谷spring aop详解
    Spring Boot 2.x Redis多数据源配置(jedis,lettuce)
  • 原文地址:https://www.cnblogs.com/linyufeng/p/13298917.html
Copyright © 2011-2022 走看看