zoukankan      html  css  js  c++  java
  • Logstash收集日志写入Redis

         由Logstash收集日志写入Redis,再有Logstash读取Redis在写入elasticsearch

         作为日志缓存介质

         官方文档:https://www.elastic.co/guide/en/logstash/current/plugins-outputs-redis.html

    一 配置Logstash写入Redis

    1.1.1 配置logstash配置文件

    [root@localhost ~]# cat /etc/logstash/conf.d/nginx.conf 
    input {
          file {
              path => "/opt/vhosts/fatai/logs/access_json.log"
              start_position => "beginning"
              type => "nginx-accesslog"
              codec => json
              stat_interval => "2"          
          }
          
    }
    
    
    output {
        if [type] == "nginx-accesslog" {
              redis {
                 data_type => "list"
             key => "nginx-accesslog-test"
             host => "192.168.10.240"
             port => "6379"
             db => "0"
             password => "123456"
          }
    
          }
       
    }

    1.1.2 验证配置文件并重启

    [root@localhost ~]# /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/nginx.conf -t
    WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
    Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
    Configuration OK
    [root@localhost ~]# systemctl restart logstash.service 

    1.1.3 检查redis是否有key

    二 另一台机器配置logstash读取redis文件并写入elasticsearch

    [root@DNS-Server tools]# cat /etc/logstash/conf.d/nginx.conf
    input {
      redis {
        data_type => "list"
        key => "nginx-accesslog-test"
        host => "192.168.10.240"
        port => "6379"
        db => "0"
        password => "123456"
        codec => "json"
      }
    
    }
    
    
    output {
        elasticsearch {
          hosts => ["192.168.10.10:9200"]
          index => "logstash-redis-logg-%{+YYYY.MM.dd}"
        }
    }

    elasticsearch-head验证

    作者:闫世成

    出处:http://cnblogs.com/yanshicheng

    联系:yans121@sina.com

    本文版权归作者和博客园共有,欢迎转载,但未经作者同意必须保留此段声明,且在文章页面明显位置给出原文连接。如有问题或建议,请联系上述邮箱,非常感谢。
  • 相关阅读:
    apache 虚拟主机配置(根据不同的域名映射到不同网站)
    Tortoise SVN 使用笔记
    Apache 根据不同的端口 映射不同的站点
    jquery 获取当前元素的索引值
    修改ThinkPHP的验证码类
    NetBeans无法使用编码GBK安全地打开该文件
    在win2003下apache2.2无法加载php5apache2_4.dll
    我看软件工程
    PHP函数参数传递(相对于C++的值传递和引用传递)
    Notepad++ 使用正则表达式查找替换字符串
  • 原文地址:https://www.cnblogs.com/yanshicheng/p/9442195.html
Copyright © 2011-2022 走看看