zoukankan      html  css  js  c++  java
  • 安装ELK elasticsearch+logstash+kibana日志监控系统简单配置

    安装ELK ,版本如下:
    Elasticsearch 2.3.5
    Logstash 2.3.4
    Kibana 4.5.4
    下载地址,请参考官网
     
    一、ES
    1、启动
    [root@elasticseach1 bin]# ./elasticsearch start
    ERROR: Parameter [start]does not start with --
     
    [root@elasticseach1 bin]# ./elasticsearch
    Exception in thread "main" java.lang.RuntimeException: don't run elasticsearch as root.
    at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:93)
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:144)
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:270)
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
    Refer to the log for complete error details.
     
    解决方法1:
    在执行elasticSearch时加上参数-Des.insecure.allow.root=true,完整命令如下:
    ./elasticsearch -Des.insecure.allow.root=true  
     
    解决办法2:
    用vi打开elaticsearch执行文件,在变量ES_JAVA_OPTS使用前添加以下命令:
    ES_JAVA_OPTS="-Des.insecure.allow.root=true"  
     
    修改后,./elasticsearch 启动成功
     
    2、修改elasticsearch.yml
    cluster.name 和 node.name
    以及 network.host 为服务器ip
     
    3、安装 elasticsearch 的插件head:
    安装:
    ./elasticsearch/bin/plugin install mobz/elasticsearch-head
    访问:
     
     
    彻底解决启动elasticSearch 时,建议不要用root 用户启动的warning :
    由于ElasticSearch可以接收用户输入的脚本并且执行,为了系统安全考虑,
    建议创建一个单独的用户用来运行ElasticSearch
     
    1、创建elsearch用户组及elsearch用户
    groupadd elsearch
    2、更改elasticsearch文件夹及内部文件的所属用户及组为elsearch:elsearch
    useradd elsearch -g elsearch -p elasticsearch
    3、屏蔽掉 bin/elasticsearch 文件的 ES_JAVA_OPTS="-Des.insecure.allow.root=true"
    4、给/elasticsearch/logs 和 data 里面的文件可写权限 chmod -R 777 logs,chmod -R 777 data
    5、切换到elsearch 用户, su elsearch ,再运行
     
    二、Logstash:
    1、bin 目录下新建 etc 目录,
    vi logstash/etc/logstash_agent.conf
    input {
      file {
        type => "nginx.access"
        path =>["/data/nginx/logs/access.log"]
      }
    }
     
    output {
      elasticsearch {
        hosts => ["10.100.100.60:9300"]
      }
    }
     
    2、启动
     
    [root@elasticseach1 bin]# ./logstash -f etc/logstash_agent.conf
    Settings: Default pipeline workers: 2
    The server failed to respond with a valid HTTP response {:class=>"Manticore::ClientProtocolException", :level=>:error}
    Pipeline main started 报错
     
    vi logstash/etc/logstash_agent.conf 修改为:
    input {
      file {
        type => "nginx.access"
        path =>["/data/nginx/logs/access.log"]
      }
    }
     
    output {
      # stdout{}
      elasticsearch {
        hosts => ["10.100.100.60:9200"]
        index => "test_output-%{type}-%{+YYYY.MM.dd}"
      }
    }
    重新启动,成功
    Settings: Default pipeline workers: 2
    Pipeline main started
     
     
    三、kibana
    1、修改kibana.yml 里面的
    server.host ,elasticsearch_url 以及 去掉 kibana.index 的注释
    ./kibana 启动
     
     
    四、kibana 连接es 索引
    1、导入json数据到es 中
    curl -XPOST '10.100.100.60:9200/shakespeare/_bulk?pretty' --data-binary @shakespeare.json
     
     
    2、从redis 中导入数据到es
    # 10.100.100.60:6379 ,成功
    input {
      redis {
        host => "10.100.100.60"
        type => "redis-input"
        data_type => "list"
        key => "elk_data"
      }
    }
     
    output {
      elasticsearch {
        hosts => ["10.100.100.60:9200"]
        index => "logstash-%{type}-%{+YYYY.MM.dd}"
      }
    }
     
     
    从 两台redis 里面导数据
    input {
      redis {
        host => "10.100.100.60"
        type => "redis-60-input"
        data_type => "list"
        key => "elk_data"
      }
      redis {
        host => "10.100.100.35"
        type => "redis-35-input"
        data_type => "list"
        key => "elk_data"
        }
    }
     
    output {
      elasticsearch {
        hosts => ["10.100.100.60:9200"]
        index => "logstash-%{type}-%{+YYYY.MM.dd}"
      }
    }
     
    要注意 redis 能连接上
    以上的服务启动都不是后台启动,后台启动请加上 &
  • 相关阅读:
    ubuntu下mysql的安装
    useradd和adduser的区别
    C和指针之学习笔记(6)
    C和指针之学习笔记(5)
    C和指针之学习笔记(4)
    Centos 7搭建Gitlab服务器超详细
    .NET Core sdk和runtime区别
    .NET平台历程介绍
    GitLabCICD
    Jenkins+gitlab+msbuild
  • 原文地址:https://www.cnblogs.com/wjq310/p/5828950.html
Copyright © 2011-2022 走看看