logstash urldecode filter 插件

Urldecode filter plugin

设置                           输入类型             	
	

field                           字符串

[elk@node01 conf]$ cat t6.conf 
input {
   stdin{}
}

filter {  
    grok {  
        match => ["message", "%{IPORHOST:ipaddress}s*(?<aaaa>([a-z]+S+[0-9]+))s*(?<bbbb>([0-9]+S+[a-z]+)).*"]  
    } 
}
output {  
        stdout {  
            codec => rubydebug  
        }  
      }


[elk@node01 conf]$ logstash -f t6.conf 
Settings: Default pipeline workers: 4
Pipeline main started
10.2.3.4 a%E6%B3%B0%E9%9A%86999 99998%E6%B5%8B%E8%AF%95bbb
{
       "message" => "10.2.3.4 a%E6%B3%B0%E9%9A%86999 99998%E6%B5%8B%E8%AF%95bbb",
      "@version" => "1",
    "@timestamp" => "2018-07-28T06:34:12.556Z",
          "host" => "node01",
     "ipaddress" => "10.2.3.4",
          "aaaa" => "a%E6%B3%B0%E9%9A%86999",
          "bbbb" => "99998%E6%B5%8B%E8%AF%95bbb"
}

	
add_fileds  

[elk@node01 conf]$ cat t6.conf 
input {
   stdin{}
}

filter {  
    grok {  
        match => ["message", "%{IPORHOST:ipaddress}s*(?<aaaa>([a-z]+S+[0-9]+))s*(?<bbbb>([0-9]+S+[a-z]+)).*"]  
    } 
 mutate {
  add_field =>["eeeee","ffffff"]
 }
}
output {  
        stdout {  
            codec => rubydebug  
        }  
      }
	  
	  

[elk@node01 conf]$ logstash -f t6.conf  
Settings: Default pipeline workers: 4
Pipeline main started
10.2.3.4 a%E6%B3%B0%E9%9A%86999 99998%E6%B5%8B%E8%AF%95bbb
{
       "message" => "10.2.3.4 a%E6%B3%B0%E9%9A%86999 99998%E6%B5%8B%E8%AF%95bbb",
      "@version" => "1",
    "@timestamp" => "2018-07-28T06:39:39.207Z",
          "host" => "node01",
     "ipaddress" => "10.2.3.4",
          "aaaa" => "a%E6%B3%B0%E9%9A%86999",
          "bbbb" => "99998%E6%B5%8B%E8%AF%95bbb",
         "eeeee" => "ffffff"
}

	
field string:
[elk@node01 conf]$ cat t6.conf 
input {
   stdin{}
}

filter {  
    grok {  
        match => ["message", "%{IPORHOST:ipaddress}s*(?<aaaa>([a-z]+S+[0-9]+))s*(?<bbbb>([0-9]+S+[a-z]+)).*"]  
    } 
 mutate {
  add_field =>["eeeee","ffffff"]
 }
  urldecode{
  field=>[aaaa]
}
}
output {  
        stdout {  
            codec => rubydebug  
        }  
      } 
	  

[elk@node01 conf]$ logstash -f t6.conf 
Settings: Default pipeline workers: 4
Pipeline main started
10.2.3.4 a%E6%B3%B0%E9%9A%86999 99998%E6%B5%8B%E8%AF%95bbb
{
       "message" => "10.2.3.4 a%E6%B3%B0%E9%9A%86999 99998%E6%B5%8B%E8%AF%95bbb",
      "@version" => "1",
    "@timestamp" => "2018-07-28T06:42:18.906Z",
          "host" => "node01",
     "ipaddress" => "10.2.3.4",
          "aaaa" => "a泰隆999",
          "bbbb" => "99998%E6%B5%8B%E8%AF%95bbb",
         "eeeee" => "ffffff"
}

	
	
Invalid setting for urldecode filter plugin:

  filter {
    urldecode {
      # This setting must be a string
      # Expected string, got ["aaaa", "bbbb"]
      field => ["aaaa", "bbbb"]
      ...
    }
  } {:level=>:error}	
	
  urldecode{
  field=>[aaaa,bbbb]
}

这样写是不行的 

[elk@node01 conf]$ cat t6.conf 
input {
   stdin{}
}

filter {  
    grok {  
        match => ["message", "%{IPORHOST:ipaddress}s*(?<aaaa>([a-z]+S+[0-9]+))s*(?<bbbb>([0-9]+S+[a-z]+)).*"]  
    } 
 mutate {
  add_field =>["eeeee","ffffff"]
 }
  urldecode{
  field=>aaaa
}
}
output {  
        stdout {  
            codec => rubydebug  
        }  
      } 
[elk@node01 conf]$ logstash -f t6.conf 
Settings: Default pipeline workers: 4
Pipeline main started
10.2.3.4 a%E6%B3%B0%E9%9A%86999 99998%E6%B5%8B%E8%AF%95bbb
{
       "message" => "10.2.3.4 a%E6%B3%B0%E9%9A%86999 99998%E6%B5%8B%E8%AF%95bbb",
      "@version" => "1",
    "@timestamp" => "2018-07-28T06:47:46.966Z",
          "host" => "node01",
     "ipaddress" => "10.2.3.4",
          "aaaa" => "a泰隆999",
          "bbbb" => "99998%E6%B5%8B%E8%AF%95bbb",
         "eeeee" => "ffffff"
}


[elk@node01 conf]$ cat t6.conf 
input {
   stdin{}
}

filter {  
    grok {  
        match => ["message", "%{IPORHOST:ipaddress}s*(?<aaaa>([a-z]+S+[0-9]+))s*(?<bbbb>([0-9]+S+[a-z]+)).*"]  
    } 
 mutate {
  add_field =>["eeeee","ffffff"]
 }
  urldecode{
   all_fields=>true
}
  
}
output {  
        stdout {  
            codec => rubydebug  
        }  
      } 
[elk@node01 conf]$ logstash -f t6.conf 
Settings: Default pipeline workers: 4
Pipeline main started
10.2.3.4 a%E6%B3%B0%E9%9A%86999 99998%E6%B5%8B%E8%AF%95bbb
{
       "message" => "10.2.3.4 a泰隆999 99998测试bbb",
      "@version" => "1",
    "@timestamp" => "2018-07-28T06:51:16.625Z",
          "host" => "node01",
     "ipaddress" => "10.2.3.4",
          "aaaa" => "a泰隆999",
          "bbbb" => "99998测试bbb",
         "eeeee" => "ffffff"
}