最近grok_exporter 发布了1.0 版本目前还是rc ，但是很期待的对于多log 文件的支持已经有了，很不错
以下是一个简单的试用
环境准备
- v1.0.0.RC2 docker 镜像
FROM ubuntu:16.04
LABEL Name="grok_exporter"
LABEL Version="1.0.0.RC2"
ENV GROK_ARCH="grok_exporter-1.0.0.RC2.linux-amd64"
ENV GROK_VERSION="v1.0.0.RC2"
RUN apt-get update -qqy
&& apt-get upgrade -qqy
&& apt-get install --no-install-recommends -qqy
wget unzip ca-certificates
&& update-ca-certificates
&& wget https://github.com/fstab/grok_exporter/releases/download/$GROK_VERSION/$GROK_ARCH.zip
&& unzip $GROK_ARCH.zip
&& mv $GROK_ARCH /grok
&& rm $GROK_ARCH.zip
&& apt-get --autoremove purge -qqy
wget unzip ca-certificates
&& rm -fr /var/lib/apt/lists/*
RUN mkdir -p /etc/grok_exporter
RUN ln -sf /etc/grok_exporter/config.yml /grok/
WORKDIR /grok
CMD ["./grok_exporter", "-config", "/grok/config.yml"]
- docker-compose 文件
version: "3.7"
services:
grafana:
image: grafana/grafana
ports:
- "3000:3000"
prometheus:
image: prom/prometheus
volumes:
- "./prometheus.yml:/etc/prometheus/prometheus.yml"
ports:
- "9090:9090"
grok:
image: dalongrong/grok-exporter:v1.0.0.RC2
volumes:
- "./example:/opt/example"
- "./grok.yaml:/grok/config.yml"
ports:
- "9144:9144"
- prometheus 配置
prometheus.yml:
scrape_configs:
- job_name: grok
metrics_path: /metrics
scrape_interval: 10s
scrape_timeout: 10s
static_configs:
- targets: ['grok:9144']
- 一个简单的grok 配置
说明次文件来自官方文档，配置日志内容，添加label 主要提取人员名称
global:
config_version: 2
input:
type: file
path: /opt/example/*.log
readall: true
grok:
patterns_dir: ./patterns
metrics:
- type: counter
name: grok_example_lines_total
help: Counter metric example with labels.
match: '%{DATE} %{TIME} %{USER:user} %{NUMBER}'
labels:
user: '{{.user}}'
server:
port: 9144
- 多日志格式说明
目前使用path 使用统配符可以，使用paths 更推荐，具体可以参考 https://github.com/fstab/grok_exporter/blob/master/CONFIG.md - log 格式说明
30.07.2016 14:37:03 alice 1.5
启动&&使用
- 构建镜像
docker build -t dalongrong/grok-exporter:v1.0.0.RC2 .
- 启动docker-compose服务
docker-compose up -d
- 追加log 信息
mkdir -p example
touch example/rong.log
touch example/examples.log
echo "30.07.2017 14:37:03 alice 1.5" >> example/rong.log
echo "30.07.2018 14:38:03 alice 1.5" >> example/rong.log
echo "30.07.2019 14:39:03 rong 1.5" >> example/examples.log
- metrics 信息
- grafana dashboard
dashboard 文件在https://github.com/rongfengliang/grok-exporter-multi-logs-learning/blob/master/grafana/dashboard.json
可以导入次dashboard 的json 文件 
说明
以上是一个简单的试用，docker 镜像以及push dockerhub 了，可以直接使用，后期应该也会更新的
参考资料
https://github.com/fstab/grok_exporter/blob/master/CONFIG.md
https://github.com/rongfengliang/grok-exporter-multi-logs-learning
https://hub.docker.com/repository/docker/dalongrong/grok-exporter