logstash 读取多个系统相同文件shipper端

input {
        file {
                type => "weblogic_10.4.32.48_access"
                path => ["/data01/zzpt/weblogic-10.4.32.48-access*"]
                start_position=>"beginning"
                codec=>plain {
                 charset=>"GBK"
               }
        }
    
       file { 
                type => "weblogic_10.4.32.111_access" 
                path => ["/data01/hdxt/weblogic-10.4.32.111-access*"] 
                  start_position=>"beginning"
                codec=>plain {
                 charset=>"GBK"
               }
        } 

 
}



filter {
        mutate {
                       add_field =>["messages","%{type}-%{message}"]
                       remove_field =>["message"]
                }


    grok {
        match =>[ 
             "messages","s*%{IPORHOST:clientip}s*-s*-s*[(?<time>(S+s+).*?)]s+"%{WORD:verb}s+%{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}" %{NUMBER:http_status_code} %{NUMBER:bytes}"
        ]
    } 
   # mutate {
   #      remove_field =>["messages"]
   #  }
}

output {
        redis {
                host => "10.4.145.93"
                data_type => "list"
                key => "%{type}:redis"
                port=>"6379"
                password => "1234567"
        }
         stdout {
                        codec => rubydebug
                } 
}
  
  
  
  
测试logstash 配置；
[elk@rsyslog logstash]$ logstash -f logstash_weblogic_access_log.conf
  
 127.0.0.1:6379> keys *
(empty list or set)


模拟数据:
[elk@rsyslog zzpt]$ echo "10.4.32.48test001" >>weblogic-10.4.32.48-access.2017-06-29

127.0.0.1:6379> keys *
1) "weblogic_10.4.32.48_access:redis"

127.0.0.1:6379> keys *
1) "weblogic_10.4.32.48_access:redis"

127.0.0.1:6379> BLPOP "weblogic_10.4.32.48_access:redis" 0
1) "weblogic_10.4.32.48_access:redis"
2) "{"@version":"1","@timestamp":"2017-07-05T00:31:52.649Z","path":"/data01/zzpt/weblogic-10.4.32.48-access.2017-06-29","host":"rsyslog","type":"weblogic_10.4.32.48_access","messages":"weblogic_10.4.32.48_access-10.4.32.48test001","tags":["_grokparsefailure"]}"


[elk@rsyslog hdxt]$ echo "10.4.32.111--tlcb999" >> weblogic-10.4.32.111-access.2017-03-18

127.0.0.1:6379> keys *
(empty list or set)
127.0.0.1:6379> keys *
1) "weblogic_10.4.32.111_access:redis"
127.0.0.1:6379> LLEN "weblogic_10.4.32.111_access:redis"
(integer) 1
127.0.0.1:6379> brpop "weblogic_10.4.32.111_access:redis" 0
1) "weblogic_10.4.32.111_access:redis"
2) "{"@version":"1","@timestamp":"2017-07-05T00:37:25.443Z","path":"/data01/hdxt/weblogic-10.4.32.111-access.2017-03-18","host":"rsyslog","type":"weblogic_10.4.32.111_access","messages":"weblogic_10.4.32.111_access-10.4.32.111--tlcb999","tags":["_grokparsefailure"]}"