zoukankan      html  css  js  c++  java
  • goaccess 通过jsonpath 转换为prometheus metrics

    goaccess 是一个不错的日志分析工具,包含了json 数据同时支持基于websocket 的实时数据处理,当然我们可以通过jsonpath
    的exporter 转换为支持promethues 的metrics,还是很方便的

    环境准备

    • docker-compose 文件
    version: "3"
    services:
      metrics:
       image: sunbird/prometheus-jsonpath-exporter
       volumes: 
       - "./conf/config.yml:/etc/prometheus-jsonpath-exporter/config.yml"
       ports:
       - "9158:9158"
       command: /etc/prometheus-jsonpath-exporter/config.yml
      g:
        image: grafana/grafana
        ports:
        - "3000:3000"
      p:
        image: prom/prometheus
        volumes:
        - "./conf/prometheus.yml:/etc/prometheus/prometheus.yml"
        ports:
        - "9090:9090" 
      log:
        image: dalongrong/goaccess-geo
        ports:
        - "7890:7890"
        volumes: 
        - "./data:/srv/data"
        - "./logs:/srv/logs"
        - "./html:/srv/report"
        - "./geoip/:/src/geoip/"
      web:
        image: nginx
        ports: 
        - "8080:80"
        volumes: 
        - "./html/:/usr/share/nginx/html/"
    • prometheus 配置

      conf/prometheus.yml

    scrape_configs:
      - job_name: go_access
        metrics_path: /
        static_configs:
          - targets: ['metrics:9158']
    • jsonpath exporter 配置

      conf/config.yml

    exporter_port: 9158 # Port on which prometheus can call this exporter to get metrics
    log_level: info
    json_data_url: http://web/app.json # Url to get json data used for fetching metric values
    metric_name_prefix: go_access # All metric names will be prefixed with this value
    metrics:
    - name: total_request # Final metric name will be go_access_total_request
      description: Number of total request
      path: $.general.total_requests
    • goaccess 配置

      主要是添加支持json 的处理data/goaccess.conf

    geoip-database /src/geoip/GeoLiteCity.dat
    log-format COMBINED
    real-time-html true
    log-file /srv/logs/access.log
    output /srv/report/index.html
    output /srv/report/app.json
    • 说明
      metrics 很简单,就是通过jsonpath 获取请求的总数

    运行&&测试

    • 启动
    docker-compose up -d
    • 效果
      goaccess

      jsonpath exporter metrics

      prometheus

    说明

    这个只是一种简单的处理实际上我们可以基于其他的nginx 或者logger exporter 进行prometheus metrics 的处理,对于需要进行离线日志请求分析的,
    也是一种不错的方案,需要的操作比较少,但是可以帮助我们分析好多有用的信息

    参考资料

    https://github.com/project-sunbird/prometheus-jsonpath-exporter
    https://github.com/rongfengliang/goaccess-geoip-docker-compose-demo
    https://goaccess.io/
    https://goessner.net/articles/JsonPath/index.html#e2

  • 相关阅读:
    Android APK瘦身方法小结
    快速了解Android重要机制
    Android 画笔Paint
    android 图片凸出
    金钱转换
    WPF属性与特性的映射(TypeConverter)
    XMAL 中x名称控件的Auttribute
    AtCoder Grand Contest 012 B
    scau 17967 大师姐唱K的固有结界
    Centos7开机自动启动服务和联网
  • 原文地址:https://www.cnblogs.com/rongfengliang/p/10688965.html
Copyright © 2011-2022 走看看