zoukankan      html  css  js  c++  java
  • filebeat结合mssql模板

    【0】下载

    注意下载的版本一定要和 ES KIBANA一样啊!

    百度搜索最佳,第一个是官方文档,第二个是下载界面

      

      

    如上图,有msi 和 zip,随便哪个都行吧;

    【1】msi版本

    我们下载了这个:filebeat-7.14.1-windows-x86_64.msi

    【1.1】安装

    傻瓜式安装,不用解释了吧?

    安装完会问你是否打开filebeat 数据与配置目录

      

    也就是说它有2个目录:

      

    【2】zip解压安装=》推荐

    (2.1)上传下载解压

    一、从官网选择对应版本下载安装包
    https://www.elastic.co/cn/downloads/past-releases#filebeat

    二、上传至目标windows主机某个盘的根目录,解压

      

    (2.2)安装 filebeat 为服务

    #进入filebeat的路径,具体看你自己放在哪里
    cd C:softwarefilebeat-7.15.0-windows-x86_64filebeat-7.15.0-windows-x86_64
    
    #进入powershell模式,如果成功,行首位置会有PS字样
    powershell 
    
    #执行安装脚本,安装filebeat服务
    .install-service-filebeat.ps1
    
    #启动mssql模块
    .filebeat.exe modules enable mssql

    【3】修改配置文件 filebeat.yml

    (3.1)修改 kibana、ouput.elasticsearch、elasticsearch template

    # ============================== Filebeat inputs ===============================
    filebeat.inputs:
    - type: log
      enabled: false
      paths:
        - /var/log/*.log
    # ============================== Filebeat modules ==============================
    filebeat.config.modules:
      path: ${path.config}/modules.d/*.yml
      reload.enabled: true
    setup.template.settings:
      index.number_of_shards: 1
    setup.template.name: "mssql-errorlog"
    setup.template.pattern: "system-*"
    setup.template.enabled: true
    setup.template.overwrite: true
    setup.ilm.enabled: false
    # =================================== Kibana ===================================
    setup.kibana:
      host: "192.168.175.132:5601"
    # ---------------------------- Elasticsearch Output ----------------------------
    output.elasticsearch:
      hosts: ["192.168.175.132:9200"]
      index: "mssql-errorlog-%{+yyyy.MM.dd}"

    (3.2)修改 mssql 模板

    找到filebeat.exe 文件位置后,在cmd下执行;

      filebeat.exe modules enable mssql

      

    然后我们去模板目录下找,发现 mssql 后缀的 disabled 已经去掉了

      

     编辑它,如下图,把路径改成我们自己的就好

      

     比如我的就是这个:

      

    (3.3)初始化启动,查看是否有问题

    filebeat.exe -e setup 

       

    没有什么报错字样,跑完就好了;

    (3.6)调试启动

    filebeat  -e run

      

    (3.7)核验(查看采集结果)

    es-head

    查看索引

      

    查看内容:(通过 es-head)

    采集过来的是乱码 玩个锤子噢

      

    但是我们通过es实际查看,是可以看到具体信息的(kibana)

    GET mssql-errorlog-2021.10.09/_search
    {
    "_source": ["host.ip","message","@timestamp"],
    "query": {
    "match": {
    "message": "1 2"
    }
    }
    }

      

    (3.8)注意事项

    1.防火墙要开放filebeat的日志流

    2.如果在安装时出现脚本执行策略的问题,可以执行以下命令修改策略

      Set-ExecutionPolicy RemoteSigned

    安装完以后再修改回来

      Set-ExecutionPolicy Restricted

    【最佳实践】

    (1)filbeat.yml 配置文件

    #=========================== Filebeat inputs =============================
    
    filebeat.inputs:
    
    # Each - is an input. Most options can be set at the input level, so
    # you can use different inputs for various configurations.
    # Below are the input specific configurations.
    
    - type: log
    
      # Change to true to enable this input configuration.
      enabled: false
    
      # Paths that should be crawled and fetched. Glob based paths.
      #paths:
      #  - C:WindowsSystem32winevtLogsApplication.evtx
      #  - C:WindowsSystem32winevtLogsSecurity.evtx
      #  - C:WindowsSystem32winevtLogsSetup.evtx
      #  - C:WindowsSystem32winevtLogsSystem.evtx
      #fields:
      #  source: system
    
    
    
    #============================= Filebeat modules ===============================
    
    filebeat.config.modules:
      # Glob pattern for configuration loading
      path: ${path.config}/modules.d/*.yml
    
      # Set to true to enable config reloading
      reload.enabled: false
    
      # Period on which files under path should be checked for changes
      #reload.period: 10s
    
    #==================== Elasticsearch template setting ==========================
    
    setup.template.settings:
      index.number_of_shards: 1
      #index.codec: best_compression
      #_source.enabled: false
    
    setup.template.name: "mssql"
    setup.template.pattern: "mssql-*"
    setup.template.enabled: true
    setup.template.overwrite: true
    setup.ilm.enabled: false
    
    
    
    
    #============================== Kibana =====================================
    setup.kibana:
      host: "115.238.30.132:5601"
    
    
    #============================= Elastic Cloud ==================================
    
    # These settings simplify using Filebeat with the Elastic Cloud (
    #-------------------------- Elasticsearch output ------------------------------
    output.elasticsearch:
      # Array of hosts to connect to.
      hosts: ["115.238.30.132:9200"]
      index: "mssql-*"
      indices:
        - index: "mssql-errorlog-%{+yyyy.MM.dd}"
          when.equals:
            service.type: "mssql"
    
    
    #================================ Processors =====================================
    
    # Configure processors to enhance or manipulate events generated by the beat.
    
    processors:
      - add_host_metadata: ~
      - add_cloud_metadata: ~
      - add_docker_metadata: ~
      - add_kubernetes_metadata: ~

    (2)模板中添加自定义字段

    那么我们可以查看数据,发现mssql采集过去的数据没有IP地址等信息啊,这不是蛋疼了嘛

      

     解决参考:https://blog.51cto.com/u_15127559/2661966

    反正我看着很复杂 搞不出来

     【故障情况】

    (1)版本不一样导致的错误

      

    但和明显我们的 filebeat 版本与 es 、kibana 的版本不一样,所以记住开头的最好是一样的啊! 装载不出 kibana模板!

    所以要用一样的模板

    【参考文档】

    https://www.jianshu.com/p/c31dbc22dee2
  • 相关阅读:
    LeetCode 88. Merge Sorted Array
    LeetCode 75. Sort Colors
    LeetCode 581. Shortest Unsorted Continuous Subarray
    LeetCode 20. Valid Parentheses
    LeetCode 53. Maximum Subarray
    LeetCode 461. Hamming Distance
    LeetCode 448. Find All Numbers Disappeared in an Array
    LeetCode 976. Largest Perimeter Triangle
    LeetCode 1295. Find Numbers with Even Number of Digits
    如何自学并且系统学习计算机网络?(知乎问答)
  • 原文地址:https://www.cnblogs.com/gered/p/15320148.html
Copyright © 2011-2022 走看看