zoukankan      html  css  js  c++  java
  • logstash之filter处理中括号包围的内容

    如题,logstash之filter处理中括号包围的内容:

    $grep -v "#" config/logstash-nlp.yml
    input {
        kafka {
            bootstrap_servers => "datacollect-1:9092,datacollect-2:9092,datacollect-3:9092"
            codec => "json"
            group_id => "logstash-newtrace-nlptemp"
            topics => ["ot-nlp"]
        }
    }
    filter {
        grok {
            match => {
                "message" => "^[%{GREEDYDATA:request}]$"
            }
        }
        json {
            source => "request"
        }
        ruby {
            code => "event.set('temptime', event.get('@timestamp').time.localtime + 8*60*60);"
        }
        grok {
            match => ["temptime", "%{DATA:thedate}T%{NOTSPACE:thetime}Z"]
        }
    }
    output {
        if ([kafka][topic] =~ "^ot-nlp*") {
            if [name] == "nlp" {
                file {
                    codec => line {format => "%{request}"}
                    path => "/tmp/newtrace_nlp.log.%{thedate}"
                }
            }
        }
    }
    

    摘出来数组中的所有fields

    filter {
        grok {
            match => {
                "message" => "^[%{GREEDYDATA:request}]$"
            }
        }
        json {
            source => "request"
            remove_field => [ "message", "request" ]
        }
        split {
            field => "binaryAnnotations"
        }
        json {
            source => "[binaryAnnotations][value]"
        }
    }
    

    感谢时总的大力支持!!!

    核心就是正则!

            match => {
                "message" => "^[%{GREEDYDATA:request}]$"
            }
  • 相关阅读:
    一些集群操作以及问题查询
    .a 文件 和 so 文件
    KNN算法
    opendir函数和readdir函数内涵及用法
    tps 和 qps的区别
    Git使用(积累一些常用的命令)
    MurmurHash
    HyperLogLog
    MySQL主从不生效且无错误
    正则表达式与SQL
  • 原文地址:https://www.cnblogs.com/zhzhang/p/9406233.html
Copyright © 2011-2022 走看看