zoukankan      html  css  js  c++  java
  • logstash使用grok过滤数据

    有一段线上日志:

    2020-07-14 11:37:04.556 INFO  [com.lyf.action.PlayAction:124] - [ 播放日志 add ] userid: 0 vid: 8079245, vtime: -1
    ┏━━━━━ Debug [native.update d_stcs_month_page set vcount = ifnul...] ━━━
    ┣ SQL:  update d_stcs_month_page set vcount = ifnull(vcount,0) + 1 where deptid = ? and vyear = ? and vmonth = ?
    ┣ 参数:         [0, 2020, 7]
    ┣ 位置:         com.chaoxing.video.mobile.front.service.stcs.PageStcsThreadTask.run(PageStcsThreadTask.java:76)
    ┣ 时间:         1ms
    ┣ 更新:         [1]
    ┗━━━━━ Debug [native.update d_stcs_month_page set vcount = ifnul...] ━━━
    

    想把log打印的内容部分通过grok读取出来(前提:使用了多行合并),也就是[ 播放日志 add ] userid: 0 vid: 8079245, vtime: -1这一段,下面是过滤规则:

    filter {
      if [type] == "debug" {
        grok {
          match => ["message", "%{TIMESTAMP_ISO8601:logdate} %{LOGLEVEL:level} (?<class>.*) - (?<log>[^┏]*)(?<sql>.*)"]
        }
        date {
          match => ["logdate", "yyyy-MM-dd HH:mm:ss.SSS"]
        }
      }
    }
    

    grokdebug分析结果如下:

    {
      "logdate": [
        [
          "2020-07-14 11:37:04.556"
        ]
      ],
      "YEAR": [
        [
          "2020"
        ]
      ],
      "MONTHNUM": [
        [
          "07"
        ]
      ],
      "MONTHDAY": [
        [
          "14"
        ]
      ],
      "HOUR": [
        [
          "11",
          null
        ]
      ],
      "MINUTE": [
        [
          "37",
          null
        ]
      ],
      "SECOND": [
        [
          "04.556"
        ]
      ],
      "ISO8601_TIMEZONE": [
        [
          null
        ]
      ],
      "level": [
        [
          "INFO"
        ]
      ],
      "class": [
        [
          " [com.lyf.action.PlayAction:124]"
        ]
      ],
      "log": [
        [
          "[ 播放日志 add ] userid: 0 vid: 8079245, vtime: -1
    "
        ]
      ],
      "sql": [
        [
          "┏━━━━━ Debug [native.update d_stcs_month_page set vcount = ifnul...] ━━━"
        ]
      ]
    }
    
  • 相关阅读:
    C#编程概念系列内容索引
    Latin1_General_BIN
    C#编程概念系列(一):面向对象编程
    System.Web.HttpException: 为 ChartImg.axd 执行子请求时出错
    Excel导入数据到数据库
    C#编程概念系列(二):应用程序域
    ASP.NET 基础结构
    [博]留作证明是原创样式
    [整]网址搜集
    仅为调博客样式
  • 原文地址:https://www.cnblogs.com/linyufeng/p/13298917.html
Copyright © 2011-2022 走看看