zoukankan      html  css  js  c++  java
  • logstash之filter处理中括号包围的内容

    如题,logstash之filter处理中括号包围的内容:

    $grep -v "#" config/logstash-nlp.yml
    input {
        kafka {
            bootstrap_servers => "datacollect-1:9092,datacollect-2:9092,datacollect-3:9092"
            codec => "json"
            group_id => "logstash-newtrace-nlptemp"
            topics => ["ot-nlp"]
        }
    }
    filter {
        grok {
            match => {
                "message" => "^[%{GREEDYDATA:request}]$"
            }
        }
        json {
            source => "request"
        }
        ruby {
            code => "event.set('temptime', event.get('@timestamp').time.localtime + 8*60*60);"
        }
        grok {
            match => ["temptime", "%{DATA:thedate}T%{NOTSPACE:thetime}Z"]
        }
    }
    output {
        if ([kafka][topic] =~ "^ot-nlp*") {
            if [name] == "nlp" {
                file {
                    codec => line {format => "%{request}"}
                    path => "/tmp/newtrace_nlp.log.%{thedate}"
                }
            }
        }
    }
    

    摘出来数组中的所有fields

    filter {
        grok {
            match => {
                "message" => "^[%{GREEDYDATA:request}]$"
            }
        }
        json {
            source => "request"
            remove_field => [ "message", "request" ]
        }
        split {
            field => "binaryAnnotations"
        }
        json {
            source => "[binaryAnnotations][value]"
        }
    }
    

    感谢时总的大力支持!!!

    核心就是正则!

            match => {
                "message" => "^[%{GREEDYDATA:request}]$"
            }
  • 相关阅读:
    AES加解密
    redis HyperLogLog使用
    vi常用操作
    redis string使用
    用jdk命令定位java应用问题
    用户态、内核态及零拷贝
    缓存穿透与雪崩
    ReentrantLock、Semaphore、AQS
    redis list使用
    不同数据库取并集、交集、差集
  • 原文地址:https://www.cnblogs.com/zhzhang/p/9406233.html
Copyright © 2011-2022 走看看